top of page

Data Collection, Cleansing and Databasing

The nuts and bolts of data workflow is the simple collection, cleaning and databasing of enough quality information to be able to perform meaningful analysis. In its simplest form, we are able to set up handwritten tables to record settings and conditions to be manually entered to database. We also can advise on the many software and hardware options available to generate the raw data required.

 

Once this data has been collected we are able to support or advise on the best methods to accurately combined and cleanse the data so as only true representations are recorded. While this may seem like a simple task, when data is generated from different sources many traps can be encountered including time zones, formatting, noisy or misleading data and incompatible units.

 

Finally, the development of a suitable data storage method is required so as the data can be efficiently accessed. Fortunately, most requirements can be comfortably met with the use of simple relational databases and well-known languages such as SQL. We also have some capabilities in the development of data lakes and the associated technologies such as Apache Hadoop, Hive and Pig.

IMG_1227.jpeg

Waterproof case with instruments, loadcells, displays and gopro's prepared for a small keel boat.

bottom of page