Data Access & Visualization

Ethica Blog Forum Contact Us
Search…
Learn Ethica
Getting Started
Reference
Tutorials
Use Cases
Frequently Asked Questions
Data Access & Visualization
Shortly after starting the enrollment and as participants provide the data to your study, you can access the data and start the analysis. You can review the data on the Researcher Dashboard, use Ethica's integrations to query and visualize the data, or export the data and use it in other tools. We discuss each of these below.
Activity Responses
Immediately after a participant completes an activity session, Ethica uploads the responses to the server and makes them available to your research team. At any time, you can check the status of the sessions in your study using the Sessions page:
Activity session results showing surveys completed by the participants
For survey activities, you can also check the responses on the Survey Responses page:
View the responded survey in Ethica researcher dashboard
On this page, you can also choose to export the data as CSV. This process happens in the background and you will get an email as soon as the export is completed. You can always check the status of your export, or download the data via the Data Export page.
Exporting Data
You can use the Data Export page to export collected sensor data in a different format:
Ethica Data Export page
In order to export the data, you should select one or more participants and a data source. Depending on the type of data you are exporting, Ethica offers one or more different formats. You always have the option to export as CSV, but for example, for GPS you may choose to export as KML or GPX as well. Further, you have to choose a date range for which you want to export the data. When done, click on the EXPORT button to start the data export process:
Data Export process started
Export files can be very large, sometimes up to multiple gigabytes. Ethica will send you an email when the export is ready. Then you can download the resulting file from the Data Export page. Each data export is available for 7 days. Data exports older than 7 days are removed automatically and you can create them again if needed:
Data Export process finished
Visualizing Data
Ethica allows you to access your data directly from the Researcher Dashboard, using Ethica integration with the Kibana visualization tool. You can use this tool to query your raw data and visualize them in different forms. Below we will give you a quick introduction to how you can use this tool. You can learn more in the Data Visualization via Kibana article.
To access Kibana, go to the Researcher Dashboard and from the right-side panel, click on Visualization via Kibana. You will be taken to a new page as shown below:
Accessing participant audit logs using Kibana
On the top left corner, you can choose the data table (or Index Patterns, as it is called in Kibana) you want to connect to. The names are self-explanatory, and you should be able to easily find the data index you want to explore. By default, you are connected to the participant_history index, which stores the participant audit logs:
Selecting index patterns
You also should choose a date range for which you want to see the data. By default, the date range filters only for the data that are received within the past 15 minutes. So chances are that you will see no data on the page. Here, we change this to point to the period from 2020-11-19 00:00:00.000-05:00 to 2020-11-21 00:00:00.000-05:00:
Selecting time range
As you can see, within this time window, our study has collected 20,500 GPS records:
Exploring GPS data via Kibana
You can filter the data based on any criteria, create graphs and visualizations based on this data, or create data dashboards by putting together your visualizations. We encourage you to read more about how you can use Kibana to visualize your study data in the Data Analysis documentation.
Participant Audit Logs
Regardless of the type of study you are conducting via Ethica, your study will involve a set of participants. For every participant in your study, Ethica records all of their interactions with the app and reports it in real-time, as part of the participant's audit logs. You can check these logs for each participant to help them navigate your study, or to troubleshoot the issues they might have, or just to use them at a later time as metadata during the analysis.
You can access the participant audit logs via Kibana. To do so, open the Researcher Dashboard and click on Visualization via Kibana on the right-side panel. By default, the participant_history index pattern is selected, which shows you the data for the participant audit reports. We just change the time range to point to the period from 2020-11-19 00:00:00.000-05:00 to 2020-11-21 00:00:00.000-05:00:
Viewing Participant Audit Report in Kibana
From the list of data fields on the right side, we click on user_id, device_id, app_version, and message. We also click on the Time column's title, to change the sorting order from descending (the default order) to ascending. Now our report looks like the following:
View Participant Audit Reports data in Kibana
Above you can see different events that have happened regarding this participant, such as:
Application opened.
Application closed.
A notification was added to the app because some settings were required.
A notification was removed from the app because the required settings were completed.
Study reload started and finished.
You can refer to the Participant Audit Logs for more details on what pieces of information are available, and how they can be interpreted.
Where to Go Next?
In the previous sections, we learned some basic skills on how Ethica works. We created a simple study and configured it to collect different types of sensor data. We programmed a few surveys and tried them out in the Ethica app. We configured when and how often the survey should be prompted. We learned how to use the criteria to add branching and skip patterns to our study. We explained what you should test before going to the field deployment, and what you should look out for while in the field. And lastly, we discussed how you can access your data and visualize them.
This should prepare you to start using Ethica for your study. We suggest as you work with different parts of the system, refer to the Reference Documentation from the left panel to learn the details on how the relevant features work. You can also refer to the Tutorials for different recipes and how-to articles.
If you have any questions, you can always ask that in our forum. You can also email us at [email protected].
Field Deployment
Reference
Last modified 27d ago
Copy link
Contents
Activity Responses
Exporting Data
Visualizing Data
Participant Audit Logs
Where to Go Next?