Functionalities

Server/Client Connections

Once the user has conencted with a give ID, a new hashed identifier is used by the library to keep track of the users and allow others to view who is currently and has been previously connected to the server.

All actions within the server/client are driven by events. An event is a message passing mechanims that allows a specific action to be understood throughout the stack. Moreover, these events have listeners and emitters wherever they are being generated, allowing for the who architecture to be informed on where and why something has occured.

Note

We are leveraging Node.js architecture for the listening and emition of events. More about it here.

Events

There are several implemented events implemented in the library will be further expanded in future releases. Specifically, we have:

Events	Description
Join, connection & disconnect	On connection/disconnection to the server. Triggers all the event listeners on the socket.
Validate-Username	Validating if a user is not already registered in the server.
Stream-Data	Creates a stream data listener for a server-peer response.
Peers-list	Returns all the list of peers connected to the server.
Peer-id	Returns a specific id given to a user for p2p connection.
Request-peer	Initializer for p2p connection.
Start & Update	smart-data-sharing Smart data transfer based on the user needs.
Peer-to-server	Upload data to a server from a given peer.
Get-task & task-result	Distributed computing listener for tasks.
Netcdf-reader, hdf5-reader, tiff-reader, gribb-reader	Specific data type readers with handlers in the server.

All events follow the same structure. There is a listener function on the clientside that passes arguments given through the application or the user into the client, which then emits and awaits for responses from the server.

//Example of an event emitter and listener
netCDFeventemitter = new EventEmitter()

//Submitting a task to the server
this.socket.on("netcdf-data", ({ data, filename }) => {
    this.netCDFEventHandler.emit("data", {
        data,
        filename
    })
});

//
this.socket.emit("netcdf-reader", {
    name: this.sessionID.clientName,
    socketId: socketId,
    dataPath: dataPath,
    fileBuffer: fileBuffer
});

On the server-side, the events have the same structure, with the required manipulation done and submitted back to the listener on the client.

      //Return a netCDF reader towards a peer(s)
socket.on("netcdf-reader", (peer) => {
    this.handlenetCDF(peer);
})

Note

Given the nature of the event emitters and listeners, new ones can be added by the client or developer, with specific manipulation techniques depending on the end use case.

Mechanisms

There are multiple ways to use the library through the events described before. Specifically, we have implemented different ways to interface with data by users within the server and themselves to create better data accomodations.

Addiitionally to the traditional file transfer system, the library has been developed with the following schemes: smart data transmissions and distributed tasks. These apply in both S2P and P2P services.

Smart Data Transmission

Files are transferred at a specified rate given by the user. Examples include ROI in a TIFF image, or a sequence of satellite images showcasing the movement of storm clouds in an area.

Distributed tasks

A main peer distributes tasks to be run by a group of connected peers to run either similar tasks on different datasets, or different tasks on the same data. The results are run on the clientside, removing a large portion of the effort a single peer would need to finish. The server and main peer instance keep track of blaming and task resolution.

Peer-to-Peer Data Transfer

The same functionalities can be replicated on a peer to peer connection as well, where one peer submits either a result or a task to another peer, then this result gets back to the requestor.

Note

All different modalities can be tailored to what the end case of the application is, and different functionalities can be easily added into the library on the server and client side.