About

spark-fits

This library provides two different tools to manipulate FITS data with Apache Spark:

  • A Spark connector for FITS file.
  • A Scala library to manipulate FITS file.

The user interface has been done to be the same as other built-in Spark data sources (CSV, JSON, Avro, Parquet, etc).

Currently available:

  • Support for Scala 2.11 & Scala 2.12
  • Read fits file and organize the HDU data into DataFrames.
  • Automatically distribute bintable rows over machines.
  • Automatically distribute image rows over machines.
  • Automatically infer DataFrame schema from the HDU header.

Support

LAL UPSUD CNRS