Data Processing
As a company with huge experience in data processing and quality analysis, mostly in the field of hydrogeology and environmental issues, we are investing a lot of our intellectual assets in order to collect the data and its quality analysis. This process requires an expert with a wide range of technical, adaptive and interpretative skills, including understanding of data mining and knowledge discovery processes, databases and warehouses, SQL and programming languages, Total Quality Management, DQ Cost and Benefits Analysis, and Data Error Detection processes.
It is valuable to mention the primary data processing activities, a set of the processing steps required to prepare the data for storage in the repository or archive from which they will be available for use in the medium term or long term. These processes are playing a big role in our business and in quality of this service, where we have developed specific primary processes, used in fields of climatology, water resource management, and other environmental areas.
Environmental Data Processing
Before reviewing the data processing tasks, it is useful to consider the means by which most environmental data are observed and recorded, because this has a significant impact on subsequent operations. The wide range of environmental variables and their dynamic nature have resulted in the situation whereby the majority of the primary environmental data are obtained from one of two sources. Our team is using the data collected specifically from the stations set on the problematic area, mostly collected with remote sensing tools. The data quality can be checked by simple data comparison with the results collected from another station.
The complete process is not that simple – it requires well-structured quality analysis process:
- Identification and retention of original records
- Data adjustments for known errors
- Aggregation and interpolation of data
- Computation of derived variables
- Data status
Once when all the steps have been done in the best manner, the data is finalized for usage.
Environmental data where we use data processing and quality analysis are:
- Climatological data
- Streamflow data
- Ground and underground water level data
- Water-quality data
- Land usage data
- Landslides data
- Floods data and many other
Quality Analysis and Validation
The comparison of the test values against the input environmental data, data checking, and data quality control represent a unique procedure called data validation. Our team is highly trained for this procedures because it has a big impact on the results and service that we are providing. There is a created commitment to perform set of the simple, complex and automated checks at all stages in the data processing and storage path because these processes are key to successful quality analysis.
Our team mostly uses the automated check, that includes 3 following steps:
- Expected ranges of variables
- Maximum expected change in a variable between successive observations
- Maximum expected difference in variables between adjacent stations
“The universal aim is to ensure the highest possible standard of all the data before they are given to our clients and users.”
For more information about this service, please,
