Skip to main content

Data

Throughout the lifecycle of the library, there have been implemented several mechanisms to deal with data commonly found within web applications and programs for the environmental and hydrological domain. This data varies based on the source and are msotly standardized to include different ways to extract and utilize it.

Data Types​

The library can handle data from multiple resources as binary streams that are moved from one point to another within the application stack. However, the main handling mechanisms added to the library include the following:

  • HDF5: The High Performance Data Format, standardized acrossed multiple domains.
  • NetCDF: Network Common Data Format.
  • TIFF/GeoTIFF: Geographically-tailored TIFF format from the OGC.
  • GRIBB: Efficiently packaged and managed data format.
  • JSON/geoJSON: Geographically-tailored JSON largely used for web mapping and locations.
  • Other commonly found image formats including PNG, JPEG, JPG.
note

The handling mechanisms current leverage a library that abstracts part of the functionalioties of the GDAL, however there is current work on the integration of a comprehensive subset of GDAL.

Data Handlers​

We have implemented different data handlers as packages by several instituions or individuals as dependencies for the library. Specifically, we have included:

  • geoTiff: small library for parsing TIFF files for visualization or analysis.
  • h5wasm: HDF5 web assembly reader/writer.
  • netCDFjs: reader and explorer for NetCDF files.
  • grib2-simple: simple javascript reader for gri2 files.

There are also aleady built-in accessors such as file readers, buffer creators, and other mechanisms for data passing and manipulation.

Note

There are new handlers being currently implemented to accomodate different ways to interface with files as well as tailor to specific needs.

All handlers are accessed through event driven interfaces that allow the users to access the hanlding mechanisms from the server. This is done through the following: 

client.gethdf5({filePath: 'path/to/files'})
//or
client.gethdf5({fileBuffer: datafileAsBuffer})
Tip

New handlers can also be implemented by utilizing the directives and frameworks established within the library. More about that in the documentation.

Note

As with other libraries, the HydroRTC library "transforms" data into small memory chunks that are afterwards interpreted throughout the application. Although multiple handlers have been implemented, the developer should make adjustements to the usage of the datasets based on the end use case required

Viewers and Analysis​

Since it is a web environment, the data coming from a server will be required to be visualized through different mechanisms. The library does not provide a specific way to create these visualizations, however, the data can be easily provided as objects or arrays into a visualization or computing library for further manipulation.

Tip

An example use on how these computing processes for visualization purposes will be shown in the case studies!

Tip

More about the data requirements here