Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Spark offers a fluent DSL with infix operators for data transforms, but after a company-imposed auto formatter got done mangling it I was only allowed to check in a wall of method chaining that none of my teammates would write.

    things
      .where($"key" === lit(key))
      .select("value")
      .as[String]
      .repartition(N1, $"value")
      .flatMap(new ThingIterator(_))
      .repartition(N2)
      .write
      .parquet(
        s"s3a://bucket/things/key=$key")
As for naming, I see stuff like “every method must take one arg whose name ends with Request” even when envelope or notification would make more sense.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: