Woopra Documentation

Data Loader Overview

Data Loader allows you to import current and historical data from external data sources for instant availability to run analytics on your data.

Features and Benefits

Data Integrity

Maintain lossless data from your database. Relying on frontend or clientside tracking can often lose data due to various factors such as blockers or incorrect coding. A direct connection to your database maintains a 1 to 1 ratio of your event data.

Historical Data

Event-based client and serverside tracking records data in real-time. However, there's no way to access historical data unless it was recorded in Woopra. When connecting to your database, we can pull any historical data and retroactively import past data and user information for immediate consumption and analysis.

No Developer Required

Often with traditional event tracking, you need a developer to code custom events on your site. With the Data Loader, all you need is the database credentials to start importing data. Setup is easy, and you can simply select what data you want to bring in -- no coding required.

What you'll need

  • Database Credentials: Required to access your database.
  • Unique Identifiers: Required so we can tie activity and visitor data to the correct user's profile.
  • Action Data: What actions are the user doing, and optionally what metadata is associated with these actions? (e.g., view page x, make a payment, use feature X).
  • Date/Timestamps: Required — we must know when the user did that particular action.

Connect Your Database

Whitelist IPs

If your data source is behind a private subnet, you might need to whitelist our server IPs so we can access the data. Here is the list of IP addresses you'll need to whitelist.

Whitelist IPs

116.202.114.34
116.202.112.173
116.202.113.228
116.202.114.101
116.202.114.102
116.202.114.105
195.201.247.43
136.243.171.42
136.243.172.140

Under the Connections section is where you'll add all your databases.

Under the Connections section is where you'll add all your databases.

  1. Navigate to the Configure tab at the top, then click Connections on the left side in the Data Loader section.
  2. Click Create Connection in the upper right.
  3. Select your integration from the dropdown and enter your credentials.
  4. Save the connection.

Schedule Your Tasks

Configure what data to bring in from your data source and when to check for updated/new data.

Check and recheck your tasks before making them active!

When you bring in new data and events, this can cause profiles to merge. While it's possible to delete events, we cannot un-merge profiles. Make sure you have set up the tasks correctly. If you are unsure, contact support@woopra.com with any questions.

  1. Navigate to the Configure tab at the top, then click Tasks on the left side in the Data Loader section.
  2. Click Create Task in the upper right.
  3. Name and configure the task (see below).

Unique Identifiers and Profile Merging

Source Configurations

Table

Select the table you want to use for importing your data. Simply select the source and navigate to the table you want to select.

Optional Table Joins

It's also possible to import data from multiple tables by configuring the relations or joins for the tables you'll be using. Joining tables allows you to pull in data from multiple tables and tells Woopra how the tables relate to each other.

  1. Select the data source you'll be using.
  2. Select the table you want to use.
  3. If you'll are using multiple tables, you can define how the tables relate by selecting a common field that's shared by both tables. This is called a primary key - foreign key relationship. For example, if you wanted to use an accounts table and a payment table, both of these tables may share an account ID or perhaps a User ID. See our example below:
Here you can see we are joining the accounts table and the paypal table. We are selecting the common filed, which is the id from the accounts table and the account_id from the paypal table.

Here you can see we are joining the accounts table and the paypal table. We are selecting the common filed, which is the id from the accounts table and the account_id from the paypal table.

Data Type

Import your data as Visitor Properties or Actions.

Identify - Visitor Properties are things like company names, emails, first and last names, locations, or any other identifying information.

Track - Action data is anything the user does that contains a timestamp. These can be things like payments, signups, subscription updates, or really any action a user can do in your application.

Track events can also update Visitor Properties.

For example say you have an 'Update Profile' event where a user can update their contact info. This could be an event, but can also update the Visitor Properties at the same time.

Select Identify to import Visitor Properties and select Track to import Action data.

Sync Interval

Select the interval for which to check for changes. You can set this as low as 1 minute.

Let's look at the following example. Say you set the task for 10 min intervals. Next, you run a task, and the import takes 1 hour. The interval waits 10 min after the initial task is complete to check for new entries. If there is new data during the next check, we'll import the new data, and once that's done, it will wait 10 min before checking again. The Sync Interval is the delay between running the task.

Tasks run queries on your database.

Each time we check for new data according to your selected Sync Interval settings, depending on your database, this could count toward your querying counts and limits. There are no querying limits in Woopra, but be sure to check with your database admins to understand your database limits or querying costs.

Conditions

If you have only certain data you want to bring in from these tables, you can use conditions to filter out the results. For example, say you were importing payment information, but you only wanted to import payments from the United States. If you have country data with the payments, you could select the 'country' column from your table and only import payments from the USA.

Mappings

Configure how to map your user properties and actions. After defining what tables and data you want to import, you now need to tell Woopra what to do with that data.

Depending on if you selected to import the data as Identify (visitor data) or Track (action/event data), the mapping configuration will change accordingly.

Select & Map User ID and User Properties: Identify

Unique IDs

This is the primary identifier to sync the data to the correct profiles. Typically this will either be a user ID or an email.

Properties

Map the columns/fields to sync with your Visitor Properties.

For example, say you have a table in your database that has the person's name, address, country, organization, etc. You can sync these fields to Visitor fields in Woopra profiles.

Here we're mapping the org_name from our database to the org_name Visitor Property in user's profiles.

Here we're mapping the org_name from our database to the org_name Visitor Property in user's profiles.

Cursor

You can think of the Cursor as a bookmark for your data that’s placed when a task completes. Once the task starts again after the set Sync Interval, it will use the Cursor as a starting point to retrieve new data.

Typically the Cursor will be a unique incrementally generated ID, a timestamp or modified date, or any unique or sequential column.

Map User Actions: Track

Select & Map User ID

This is the main identifier to sync the data to the correct profiles. Typically this will either be a user ID or an email.

Action Name

Choose a name for the action you are importing. For example, make_payment, update_subscription, new_registration, update_profile, etc.

Action Properties

Each action can have its own set of properties. For example, if you have a payment event, this may have properties such as amount, plan type, start date, upgrade, etc.

To map these fields, you'll select the column from your tables, and map those to event properties in Woopra.

Here we created the 'signup' event with the properties: plan_id, org_name, and plan_type.

Here we created the 'signup' event with the properties: plan_id, org_name, and plan_type.

Timestamp

Select the column that contains the timestamp of the event. Each event must have a timestamp associated with it.

Cursor

You can think of the Cursor as a bookmark for your data that’s placed when a task completes. Once the task starts again after the set Sync Interval, it will use the Cursor as a starting point to retrieve new data.

Typically the Cursor will be a unique incrementally generated ID, a timestamp or modified date, or any unique or sequential column.

Data Loader Overview


Data Loader allows you to import current and historical data from external data sources for instant availability to run analytics on your data.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.