Useful Data Tools
During our adventures designing, building and testing Hydrolix's platform we've had to navigate the many dimensions of data that are out there. This has led us to seek out tools that allow us to manipulate data and work with systems.
Below are a collection of tools we've found that have simply been helpful in that journey. They are not developed by us, nor do we have any kind of relationships with the companies or people who have developed them, we simply hope you will find them as useful as we have. We will keep adding to this list.
Tool | Use | Link |
---|---|---|
jq | JSON Manipulation | https://stedolan.github.io/jq/ |
Ebay Utils | CSV/TSV Manipulation | https://github.com/eBay/tsv-utils |
Re2 | Regex Testing | https://regoio.herokuapp.com/ |
Regexper | Regex "Drawing" | https://regexper.com/ |
Go DateTime Tools
Hydrolix uses Go's date parsing for ingesting datetimes into its system. The following we have found useful in checking this datetime format.
Resource | Link |
---|---|
GoTime Docs | https://golang.org/pkg/time/ |
Go Playground | https://play.golang.org/ |
The Go PlayGround can be really helpful checking your parsing of datetime formats. We'd recommend using the below to check how datetime format is being correctly parsed if you are having trouble with it.
In the Go Playground box add the following:
package main
import (
"fmt"
"time"
)
func main() {
layout := "2006-01-02T15:04:05.000+0000"
data := "2020-11-09T00:00:00.000+0000"
t, err := time.Parse(layout, data)
fmt.Println(t, err)
}
Replace Layout with your defined layout for the timestamp and data as an example datetime you wish to parse. Once clicking Run
the tool will tell you if it parses or not.
Updated about 1 year ago