WURFL Integration

Hydrolix provides automatic WURFL (Wireless Universal Resource FiLe) integration. Supply a ScientiaMobile license key and a User-Agent string in your data to automatically decorate table rows during ingest with over 600 WURFL device capabilities. This feature was introduced in version 6.4.5.

WURFL integration is provided via Hydrolix's Auto Values system and a few extra settings in the Transform.

Specify Settings

Add a "wurfl" section to the settings section of each transform where you’ll be ingesting WURFL data. Within that section, add these configuration options:

NameDescription
enabledEnables the ScientiaMobile WURFL integration.
update_urlSpecifies your ScientiaMobile-provided update URL. This included device lookup data and your license.
header_mappingTells Hydrolix which key in your data specifies the User-Agent to facilitate device lookup.

For example, in your transform:

	"settings": {
        # <your other settings here>
        "wurfl": {
            "enabled": true,
            "update_url": "https://data.scientiamobile.com/{YOUR-SCIENTIAMOBILE-LICENSE-KEY-HERE}/wurfl.zip",
            "header_mapping": {
                "User-Agent": "user_agent"
            }
        }
    }

The above configuration does the following:

  1. Ensures that WURFL is enabled.
  2. Configures the update_url to point to your licensed WURFL definition.
  3. Sets the header_mapping key to be a map of HTTP headers that WURFL will use for classification.

Include Fields

Provide a mapping in your transform schema for each field you want to include in your Hydrolix table. Here’s an example for the field "physical_screen_width".

{
  "name": "physical_screen_width",
  "datatype": {
    "type": "string",
    "index": true,
    "source": {
      "from_automatic_value": "physical_screen_width"
    }
  }
}

Notes

If your results are partially decorated, check your ScientiaMobile license level. Some fields are only available with Premium plans from ScientiaMobile.

Initially, undecorated rows are recorded until the ScientiaMobile libraries has completely loaded. This preserves Hydrolix's throughput even if the WURFL capabilities are not yet ready to use.

Example

Assuming you have simple CSV-formatted data including just a timestamp and the contents of a User-Agent header:

1698438520,"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36"
1698438522,"Mozilla/5.0 (Linux; Android 13; SM-S901U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36"
1698438524,"Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Mobile Safari/537.36,gzip(gfe)"
1698438527,"Mozilla/5.0 (Linux; Android 12; Redmi Note 9 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36"
1698438529,"Mozilla/5.0 (Linux; Android 12; DE2118) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Mobile Safari/537.36"
1698438530,"Mozilla/5.0 (iPhone12,1; U; CPU iPhone OS 13_0 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/15E148 Safari/602.1"
1698438534,"Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1"
1698438536,"Mozilla/5.0 (Apple-iPhone7C2/1202.466; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543 Safari/419.3"
1698438537,"Mozilla/5.0 (Windows Phone 10.0; Android 6.0.1; Microsoft; RM-1152) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Mobile Safari/537.36 Edge/15.15254"
1698438538,"Mozilla/5.0 (Linux; Android 11; Lenovo YT-J706X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36"
1698438541,"Mozilla/5.0 (Linux; Android 6.0.1; SGP771 Build/32.2.A.0.253; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/52.0.2743.98 Safari/537.36"
1698438543,"Roku4640X/DVP-7.70 (297.70E04154A)"
1698438546,"Mozilla/5.0 (PlayStation; PlayStation 5/2.26) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Safari/605.1.15"

The transform below will automatically use WURFL and the user_agent field to automatically decorate new fields in your tables with device information from ScientiaMobile's list of WURFL device capabilities:

{
  "name": "wurfl-test-transform-csv",
  "type": "csv",
  "settings": {
    "wurfl": {
      "enabled": true,
      "update_url": "https://data.scientiamobile.com/{YOUR-SCIENTIAMOBILE-LICENSE-KEY-HERE}/wurfl.zip",
      "header_mapping": {
        "User-Agent": "user_agent"
      }
    },
    "output_columns": [
      {
        "name": "ts",
        "datatype": {
          "type": "epoch",
          "index": false,
          "primary": true,
          "source": {
            "from_input_index": 0
          },
          "format": "seconds",
          "resolution": "second"
        }
      },
      {
        "name": "user_agent",
        "datatype": {
          "type": "string",
          "source": {
            "from_input_index": 1
          }
        }
      },
      {
        "name": "form_factor",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "form_factor"
          }
        }
      },
      {
        "name": "advertised_device_os",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "advertised_device_os"
          }
        }
      },
      {
        "name": "advertised_device_os_version",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "advertised_device_os_version"
          }
        }
      },
      {
        "name": "av1_hardware_decode_level",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "av1_hardware_decode_level"
          }
        }
      },
      {
        "name": "resolution_width",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "resolution_width"
          }
        }
      },
      {
        "name": "resolution_height",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "resolution_height"
          }
        }
      },
      {
        "name": "chipset",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "chipset"
          }
        }
      },
      {
        "name": "ram",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "ram"
          }
        }
      },
      {
        "name": "gpu_model",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "gpu_model"
          }
        }
      },
      {
        "name": "is_connected_tv",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "is_connected_tv"
          }
        }
      },
      {
        "name": "is_ott",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "is_ott"
          }
        }
      },
      {
        "name": "release_msrp",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "release_msrp"
          }
        }
      },
      {
        "name": "release_date",
        "datatype": {
          "type": "string",
          "source": {
            "from_automatic_value": "release_date"
          }
        }
      }
    ]
  }
}

The resulting rows in your Hydrolix table will contain information like this:

{
	"advertised_device_os": "Android",
	"advertised_device_os_version": "11",
	"av1_hardware_decode_level": "-1",
	"chipset": "Helio G90T",
	"form_factor": "Tablet",
	"gpu_model": "ARM Mali G76 3EEMC4",
	"is_connected_tv": "false",
	"is_ott": "false",
	"ram": "8GB",
	"release_date": "2021_june",
	"release_msrp": "548",
	"resolution_height": "1200",
	"resolution_width": "2000",
	"ts": "2023-10-27 20:38:58",
	"user_agent": "Mozilla\/5.0 (Linux; Android 11; Lenovo YT-J706X) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/96.0.4664.45 Safari\/537.36"
},