Useful Data Tools

During our adventures designing, building and testing Hydrolix's platform we've had to navigate the many dimensions of data that are out there. This has led us to seek out tools that allow us to manipulate data and work with systems.

Below are a collection of tools we've found that have simply been helpful in that journey. They are not developed by us, nor do we have any kind of relationships with the companies or people who have developed them, we simply hope you will find them as useful as we have. We will keep adding to this list.

ToolUseLink
jqJSON Manipulationhttps://stedolan.github.io/jq/
Ebay UtilsCSV/TSV Manipulationhttps://github.com/eBay/tsv-utils
Re2Regex Testinghttps://regoio.herokuapp.com/
RegexperRegex "Drawing"https://regexper.com/

Go DateTime Tools

Hydrolix uses Go's date parsing for ingesting datetimes into its system. The following we have found useful in checking this datetime format.

ResourceLink
GoTime Docshttps://golang.org/pkg/time/
Go Playgroundhttps://play.golang.org/

The Go PlayGround can be really helpful checking your parsing of datetime formats. We'd recommend using the below to check how datetime format is being correctly parsed if you are having trouble with it.

In the Go Playground box add the following:

package main

import (
	"fmt"
	"time"
)

func main() {
	layout := "2006-01-02T15:04:05.000+0000"
	data := "2020-11-09T00:00:00.000+0000"
	t, err := time.Parse(layout, data)
	fmt.Println(t, err)
}

Replace Layout with your defined layout for the timestamp and data as an example datetime you wish to parse. Once clicking Run the tool will tell you if it parses or not.