The Hypermass CLI delivers live data from streams to a folder on your machine, in real time as it's published.
1. Install
See the installation instructions (don't worry, they link back here when you're done).
If you opted just to try it rather than install, remember to type ./hypermass (or hypermass.exe for windows) in
the commands below rather than just hypermass.
2. Subscribe and sync
Try it with a live stream of solar images:
hypermass subscribe _254gGE43g
hypermass sync
Subscribe just updates the configuration. Sync starts the live porcess that fetches data, so leave hypermass sync
running. You'll see some messages appear about what sync is doing (you can kill it at any time with ctrl-C).
Files arrive in <home folder>/hypermass/subscriptions/_254gGE43g as they're published, one file per payload. In the
above example you should see the most recent image of the sun captured by the SOHO telescope
(see: EIT 284a instrument) in the folder.
If you leave hypermass sync running for a day, the next day's images will arrive in the same folder as it is
published (typically about 4:15 GMT).
To subscribe to any other stream, the Get live data button on its page at hypermass.io shows the command, ready to copy.
3. Use the data
Now that files are arriving in your local folder, you can process them and delete them when you are done. The
hypermass sync process won't modify that file again, and deleting those files once processed is a good way to keep
track of which ones you've handled (don't worry you can always replay the stream messages if you need to).
The hypermass cli keeps track of what files have been fetched, and they always arrive in the published order. Each file is named with its payload id, and sorting the files by name puts them in that same published order. We also update the last modified time of the file to the time the payload was published - but this can be less reliable since opening the file may modify this. If you need reliable timestamps, consider using the folder-with-metadata configuration.
.hypermass folderEach stream's folder holds a hidden .hypermass folder. It records which stream the folder belongs to and the last
payload received - your place in the stream. Delete only the data files, and leave .hypermass in place: without it,
the CLI no longer recognises the folder as the stream's, and your place in the stream is lost.
Next steps
- See what else it can do: check every stream's progress with
hypermass status, replay a stream from any point in its history, and change your streams on the fly withhypermass reload- Using The CLI - Put the data to work in your own code: Building a client
- Run it on a server: install
hypermass syncas a service, so it keeps running across restarts - Download & Install - Publish your own data: Publish your own streams
Who is the CLI for?
We envisage that the Hypermass CLI will be the 'usual' way to automatically synchronise data from Hypermass to your desktop and server, for use with your programs and workflows. If you are a data scientist, developer or researcher it's a convenient and repeatable means of fetching and publishing the data that you need with a single command.
For programs and production environments, it offers a neat, simple method of integrating hypermass data streams into your application.
Why this approach?
A command line tool allows for automation and also for user operation. It works for many platforms and languages. If you're interested in our thought process in creating the CLI, please see our detailed article CLI Design Rationale.
The "sync" command
The hypermass sync command is the real magic of the Hypermass cli approach - it both subscribes and publishes. It
puts data from Hypermass streams into local folders for you to process however you want, and uploads any files you
write to the folders listed under publication-sources in your config.
Once sync is running, you can use the hypermass status command in another terminal to see the current status, file
queues and other useful information.