Archive

Archive for the ‘Managing Projects, Data & The Organization’ Category

How To Flag Interviews Based On Duration

October 13, 2015 Leave a comment

Overview

The duration is one of the most important variables when doing quality control of interviews, along with Silent Recordings and GPS Capturing. Reviewing the duration allows you to quickly detect suspicious interviews as if you expect an interview to take ~30 minutes yet the interview took ~4 minutes immediately you know this interview needs to be reviewed. Instead of manually scanning for interviews which have short (or high) durations we have added an ability to have SurveyToGo automatically flag interviews that took less than a certain amount of minutes or more than a certain amount of minutes. To enable this:

Step 1: Configure the Quality Control duration flagging for the relevant survey

Step 2: Use the Operations Console to spot flagged interviews

Step 3: Exporting the Quality Control flag and comment

 

Step 1: Configure the Quality Control duration flagging for the relevant survey

To configure the survey to flag duration related situations:

  1. Switch to the Quality-Control tab of the survey:
  2. Next, check the relevant box, and specify the minutes. For example, to flag interviews that took less than 4 minutes:
  3. Save the survey and verify that the interviewers have synched the survey to the latest version.

Important Note: The flagging will start happening as soon as the survey is synched to the devices. Devices that have not synched to the version where you have activated the flagging in will not flag the interviews so make sure the interviewers are re-synching after you make this change.

 

Step 2: Use the Operations Console to spot flagged interviews

Once interviews are flagged and uploaded, you can use the Operations Console to immediately spot these interviews. To view the flagged interviews:

  1. Open the Operations Console of the relevant project and click the “Get” button to pull the interviews:

  2. Next, on the grid you can see the “Quality Control Flag” column and “Quality Control Flag Comment” appear. For interviews that were flagged by duration you will see the following values:

    So now you can immediately spot from the grid the relevant interviews that were flagged. If you click the “Quality Control” flag you can order the interviews based on the flag status and if you drag the “Quality Control Flag” column to the “Group By” area you will be able to group the interviews by the flag type. For example:

    In our example you can see 1 interview in the duration group and 27 not flagged:

  3. In addition, if you double click the flagged interview you will be able to see a visual alert that the interview was flagged on the top of the review screen:

 

Step 3: Exporting the Quality Control flag and comment

You can also choose to export the Quality Control Flag and comment along with your data in case you need to process outside of the SurveyToGo solution. To do so, during the export wizard:

  1. On the options tab, select the Quality Control flag and comment field from the list of the available fields and click the “Right arrow” button to move them over to the list of exported fields:
  2. Then continue with the export process as usual.
  3. The new exported fields will be included in the output data file.

That’s it!

Visit us at: http://www.dooblo.net

How To: Challenges of running Tracker Projects with Fixed-File formats like SPSS Quantum and Triple-S, Dooblo Recommendations

August 25, 2015 Leave a comment

Overview


Most Market Research projects tend to be either an Adhoc type of project or a Tracker type of project. The main difference between the two being that while with adhoc studies the data is usually only exported once at the end of the project, the data of tracker based studies is exported multiple times, usually after each wave is complete and in order for the data processing team to be able to tie up the new wave data to the old one(s) there might be more challenges than with the case of adhoc studies. This guide covers the challenges of using fixed file formats like SPSS Quantum or Fixed Triple-S with tracker type of studies. Dooblo essentially recommends to not use these formats with tracker based studies unless you are fully aware of the challenges and know exactly how to handle them.

 

The Challenges of fixed file formats & trackers

The big issue of fixed file formats is that they are, well, fixed! So if the coding of Q1 needs to be in row 1, column 1 of the file and takes up 1 character – it needs to stay that way throughout the entire lifecycle of the tracker data and must never change otherwise it will generate issues when aggregating multiple data files from multiple waves. Here are some examples of common scenarios that can cause a huge issue with trackers data files that are later aggregated:

  • Single choice question with 9 brands to choose from, which after a few weeks and a few addition suddenly now has 12 brands: So in the first weeks 1 character was enough to hold the code (1-9), now suddenly 2 characters are needed.
  • Grid question with 6 topics now suddenly has 7 topics: So again, the data needs to be shifted somehow to accommodate the extra topic.
  • A question is added between 2 existing questions: So now all the previous data needs to be shifted to the correct position to accommodate the change, or other measures need to be taken.

These are just 3 examples of common scenarios that happen frequently with tracker studies and the potential effect on the Data Processing team might be critical as it might cause a lot of extra work to re-generate the previous data in the correct format to allow for the data aggregation that is needed.

 

Possible solutions

Actually, the best solution to tackle these challenges is to not face them to begin with. This means that it is our recommendation to not use fixed file formats with tracker studies due to the potential headache that can be caused by these older formats. The newer formats like SPSS and SPSS Data Collection (formerly known as Dimensions) do not depend on the specific position of the data within the file but rather refer to the data based on variable names and codes which means the above challenges will not exist. However, we recognize that in some cases usage of the fixed file formats like SPSS Quantum or Fixed Triple-S can’t be avoided and therefore we have constructed a set of best-practices for using these types of formats for tracker studies:

  • Make sure to always leave “spaces” for answers. For example, if you have less than 10 answers for a question, make sure to add 20 answers and make the last 10 invisible so that 2 characters are saved for the answers.
  • The same goes for topics in grids. Always make sure to add a few extra topics so that more space is reserved in the data file to accommodate future topics.
  • When adding questions, make sure to manipulate the maps of the data file so that the question is exported at the end of the file and not in the natural position

The above best practices will tackle the challenges by ensuring extra space is saved in the file to accommodate future changes in the script. However the above are not a 100% guarantee as more radical changes in the script can still cause issues.

 

SPSS Quantum specific challenges

In addition to the above mentioned challenges, as SPSS Quantum is a very old format which for quite some time now is being phased out by IBM (see this page for end-of-support info), Dooblo has stopped adding support for newer question types (like multiple selection gird and location questions) to the Quantum exporter meaning you should consult with Dooblo support to ensure your script only uses supported question types. Please note that Dooblo recommends using the Quantum format only in extreme cases where other formats can’t be used.

 

Fixed file Triple-S specific challenges

With Triple-S Dooblo has added an export mode called “enhanced Triple-S” which is intended to be used with trackers. In this mode SurveyToGo will automatically “space out” the fixed file format to try and implement the best-practices noted above along with adding some special Dooblo XML elements to the map file in order for SurveyToGo to be able to identify the questions in the map and to accommodate the best-practices:

It is recommended to use this mode when running trackers with Triple-S.

 

Summary

The use of “Fixed File” formats such as SPSS Quantum and Triple-S for tracker study is highly not recommended due to the inherent limitations within these files and the data processing requirements of such projects. Dooblo recommends using export formats such as SPSS or IBM SPSS Data Collection (formerly Dimensions) with tracker studies. In case using filed-file formats is critical for the project, please consult the above best practices and the Dooblo support team to ensure smooth execution of the project.

Visit us at: http://www.dooblo.net

How To Use The SurveyToGo REST API

July 23, 2015 7 comments

 


Overview

 

You can access your organization data using the SurveyToGo REST API. The REST API allows you to create a tight integration between the SurveyToGo system and any 3rd party system. Using the API requires prior knowledge of the REST protocol and experience in programming. You can use the REST API from any operating system and any programming language. The SurveyToGo REST API consists of a set of operations which are listed below. Please note we are adding to the list of operations so check back often or subscribe to this topic to stay on top of any new REST operation that is supported.

 

Downloading the Testbed App and Full Source code

 

Dooblo provides you with a free SurveyToGo REST API Testbed app that allows you to test all the available operations and see the output you will get to assist with the development process. In addition we provide the full C# source code of the Testbed app so that you can see the actual implementation of the code. Please note you can call the REST API from any language, not just C#:

 

SurveyToGo REST API Operations Available

 

The following operations are available:

 

General Account management:

  • GetCreditBalance – returns the current interview balance
  • GetStorageStatus – returns the current storage space usage

 

User Administration & Assignment

  • AddSurveyorUser – adds a new interviewer user to the organization
  • DeleteSurveyorUser – deletes the interviewer user from the organization
  • AssignSurveyorUserToProject – assigns the interviewer user to the project
  • UnassignSurveyorUserFromProject – unassign the interviewer user from the project
  • GetProjectSurveyorUsers – returns the list of interviewers users assigned to a project
  • GetProjectSurveyorGroups – returns the list of interviewer groups assigned to a project
  • GetGroupUsers – returns the list of users in a group

 

Projects & Customers

  • Customers – returns the list of customers in the organization
  • CustomerProjects – returns the list of projects of a certain customer
  • ProjectSurveys – returns the list of surveys within a project
  • CreateCustomer – creates a new customer
  • CreateProject – creates a new project within a customer
  • CloneSurvey – creates a copy of an existing survey in the same or different project
  • Surveys – returns the details of a specific survey
  • GetSurveyXML – returns the full survey XML structure of a specific survey

 

Data

  • OperationData – returns the interview operational data in a form similar to the operations console
  • SurveyInterviewIDs – returns the list of interview ID’s of a survey based on filters
  • SurveyInterviewData – returns the actual XML data of the interviews based on ID’s.

 

Interviews

  • ChangeStatus – changes the status of an interview

 

Additional Data Lists

 

 

 

URL For Calling the REST API

 

The SurveyToGo REST API is hosted at this URL: https://api.dooblo.net/newapi

Calling the actual operations involves referencing the correct URL and passing the relevant user/password details.

 

User ID for Accessing the REST API

 

To call the various operations you need to specify a valid user ID. The user ID is in the form of XXXX-XXXX-XXXX/YYYY where the first part is your organization REST API Key and the second part is the relevant user name you wish to use for the operations. Please note the user accessing the REST API needs to have the relevant permissions for the operations that user is calling. For example, if the user does not belong to a certain project that user will get an authorization error if that user attempts to call operations for that project.

 

For example, if the user you wish to use when calling the REST API is named “mark” and the organization REST API Key is: f150fc49-3a14-4ecb-9214-ca04698a9578 the user ID that you need to use when calling the REST API: f150fc49-3a14-4ecb-9214-ca04698a9578/mark

Contact support@dooblo.com to receive your organization REST API Key to use with the REST API.

 

JSON vs XML Output of requests

 

In some calls you can request to receive the output of the data in either JSON or XML format. This is intended to cater to each programmer preference. However, please keep in mind that for some operations that return either interview data or survey data in them, only XML will be supported as an output format. If you try to call these with JSON as format you will receive an error: “Error: This method supports only xml” from the API.

 

Sharing of your SurveyToGo REST API Integration Success Stories

 

If you developed a cool integration using the SurveyToGo REST API, we would love to hear about it and possibly share the project highlights with the rest of our community along with your name and details (if you wish) to promote your expertise and broaden the horizon of our community! Let us know by emailing us at ofer@dooblo.com .

 

Missing an API that will assist you in your integration? Let us know!

 

If during the development of your integration you are missing an API please let us know at support@dooblo.com and we will make sure our SurveyToGo REST API product team is notified of the request to be reviewed for adding to future versions.

 

 

Visit us at: http://www.dooblo.net

 

Different ways of terminating an interview

November 16, 2014 Leave a comment

Overview

While collecting data, sometimes, an interview has to be terminated or filtered out due to the survey’s logic as indicated in the questionnaire..
SurveyToGo enables you to terminate interviews during their execution in several different methods.

In this How-To we will go over the following termination methods:

  1. Canceling interviews.
  2. Filtering interviews.

Canceling Interviews

In most questionnaires there are logic instructions that require you to terminate the interview – for example during the screening process. In many situations you will not be interested in the data collected during the screening process thus indicating SurveyToGo to not save the collected data while terminating it. This is referred to as canceling the interview.

Canceled interviews will not be saved on the device hence, are not uploaded to the server and do not show in the operations grid. Canceled interviews are not recorded nor credited from your organization credit balance.

Let’s say we have a single choice question asking for the respondent’s age group:


For Example, we would want to cancel all interviews which are conducted with respondents that are not within the 18-60 age group, we would have to add it in the ‘Jump Rules’ section of the question.
How To Add Jump Rules
Add a jump rule saying that if the answer to this age group question is other than ’18-60` , Cancel the interview:


The jump rule shown above will cancel our interview and bring us back to the ‘Start’ screen before running a new survey.
If you wish to be prompt before a survey is cancelled, you can tick this feature’s checkbox through the survey’s properties tab:


Then, when a cancelation occurs, you would be prompt as following:


* ‘Yes’ will cancel the interview.
*’No’ will get back to the question and you would be able to change the answer.

 

Filtering interviews

In many surveys you will be interested in keeping the data collected during the screening process for analysis purposes.

In such cases you will be indicating SurveyToGo to save the collected data while terminating it. This is referred to as Filtering the interview.

Filtered interviews will be saved on the device hence, will get uploaded to the server and show in the operations grid. Filtereded interviews are credited from your organization credit balance.

This is more common to use in later stages of the survey, after collecting information which you would like to save. Let’s say we have a single choice question asking for the respondent’s favorite brand (after a long questionnaire including general questions, LSM, brand awareness, etc):

 


In this case, we would like to filter all interviews which the favorite brand was not `Brand 3`. Meaning, the survey will continue from this point only if the favorite brand was `Brand 3`. For other answers, the interview will terminate at that point and the data will be saved and uploaded, indicated as filtered.

Add a jump rule saying that if the answer is other than ‘Brand 3`, Filter the interview:


Regarding prompts, it is the same as jump rules, you should tick it’s feature check box in the ‘Survey Properties’ tab:



Filtered results are not included in the exporting by default, if you wish to add them to your exporting,
you should do this manually through the 
exporting wizard:

When exporting to excel, you would be able to see which subjects were filtered out.

Each value different than (-1) is filtered, and the value represents the number of question it was filtered at.


Please note this (as mentioned above):

Canceled results are not counted in your organizations credit balance.
Filtered results do count in your organization credit balance.

That’s it!

How to: Using the Operational Dashboards Fieldwork Reports

March 26, 2014 1 comment

 

Overview

The Dashboards Fieldwork reports feature allows you to easily view various reports about your fieldwork and more importantly allows you to generate custom reports based on the operational data of your project. To use the Fieldwork reports:

 

  1. Step 1: Define the Fieldwork reports date range and survey
  2. Step 2: Run the pre-set report
  3. Step 3: Run Custom reports

 

Pricing Note: The SurveyToGo Fieldwork Reports are a part of the Dashboards feature along with the Operational Dashboards and comes at a monthly charge (unlimited use) to our pay-per-interview customers, and is included free of charge for SurveyToGo Subscriptions customers. Request your 7 days free trial of the Operational Dashboard feature today by contacting support@dooblo.com . To sign up for the SurveyToGo Dashboards, please see this link: https://www.dooblo.net/stgi/order.aspx?sku=3229012

 

Step 1: Define the Fieldwork reports date range and survey

 

To define the Fieldwork reports date range and survey:

  1. Open the project and the Dashboards node:
  2. Double click the Fieldwork Reports node and set the relevant filters:

  3. Once you are done setting the filters (dates, included interviews and the survey) click the “Refresh” to generate the Fieldwork reports

 

Step 2: Run the pre-set reports

 

After you click the “Refresh” button you will see the Fieldwork reports screen with the first pre-set report already loaded, the “Daily uploads by surveyor”:

 

 

To switch between the pre-set reports simply choose one of the reports from the “Preset Views” box:

 

 

The following pre-set reports are available, along with the “Custom” report that allows you to pivot the data and generate any custom report that you wish:

 

 

Daily uploads by surveyor

 

 

The “daily uploads by surveyor” shows you the number of uploaded interviews per interviewer for any given day along with the totals.

 

Daily Uploads by Survey versions

 

 

The “daily uploads by survey version” shows you the number of uploaded interviews for a given survey version for the time range. This report is very useful to make sure that once you deploy a new version of the survey you do not get any new interviews for the older version of that survey and thus confirm that all interviewers are running with the latest version of the survey.

 

Daily Uploads by Interview status

 

The “daily uploads by interview status” shows you the number of interviews in each of the different statuses for a given day in the time range. This report is very useful when you need to make sure that no “old interviews” have not yet been approved.

 

Daily filtered interviews

 

 

The “Daily filtered interviews” shows you the interviews of any given day, broken down by the question index that caused the interview to get filtered. -1 stands for “not filtered”. Please note that to make this report show accurate data you will need to check the “include filtered interviews” in the report filter:

 

 

GPS Usage by surveyor

 

 

The “GPS Usage by surveyor” shows you for each of the surveyors, how many interviews they completed vs the number of interviews completed with GPS information. This report is very useful to pin point which interviewer is not using the GPS feature on his/her device and get the interviewers to start using GPS.

 

Attachments uploaded by surveyor

 

 

The “Attachments uploaded by surveyor” report shows you the number of interviews uploaded by the interviewer vs the number of attachments uploaded during that day for that interviewer. The data is also broken down by day. This report is useful to confirm that interviewers are collecting attachments for surveys.

 

Step 3: Run custom reports

 

In addition to the pre-set reports, the fieldwork reports feature also includes the ability to run any custom reports on the various data fields available. To do this, first select the “Custom” out of the preset reports box:

 

 

Then, you can start dragging the relevant fields to the reports pivot area. This is very similar to the Excel Pivot tables feature, so if you are familiar with that you should feel right at home with this pivot report.

 

Let’s generate a sample custom report that will show us the status of the interviews broken down by interviewer (instead of by day as in the pre-set reports). First we’ll start with dragging the “# interviews” to the “Values” report box:

 

 

This will tell the Pivot report that we are reporting on the “# of interviews” value and will generate a report with only the “value” visible as we have yet to break it down. The value represents to total number of interviews for this date range:

 

 

 

Next, we will drag the “Surveyor” field to the “Row labels”:

 

 

 

This will tell the Pivot report to break down the number of interviews by the different interviewers and will expand the report with the breakdown of the interviewers:

 

 

The final step, is to add the cross break down of the status. To do this, drag the “Status” field to the “Column labels”:

 

 

This will add a cross break down by the status and will generate the report:

 

 

This concludes the sample custom report. Lets add a final step, and add an additional cross break down of the day. To do this, drag the UploadTime.Day field to the “Column labels”:

 

 

This will generate a 2nd level cross of the data and will break it also down by day:

 

 

As you can see you can generate any custom report on the data by dragging the relevant fields to the “Row Labels” and “Column labels”. We’ve also seen that you can drag more than one field to the row/column to generate multi level reports. You can also drag the following fields to the “Values”:

  • # of interviews
  • # of GPS (number of interviews with GPS location information)
  • Attachments (number of attachments)

 

Visit us at: http://www.dooblo.net

How to: Using the Operational Dashboards

March 26, 2014 Leave a comment

 

Overview

 

The Operational Dashboard feature allows you to easily view key performance indicators of your running survey. You can use these KPI’s to evaluate whether your project is running on track and determine the throughput of your field staff. To use the Operational Dashboard:

 

  1. Step 1: Define the Operational Dashboard date range and survey
  2. Step 2: Run the report

 

Pricing Note: The SurveyToGo Operational Dashboards are a part of the Operational pack feature along with the Fieldwork Reports and comes at a monthly charge (unlimited use) to our pay-per-interview customers, and is included free of charge for SurveyToGo Subscriptions customers (professional and up). Request your 7 days free trial of the Dashboard feature today by contacting support@dooblo.com . To sign up for the SurveyToGo Dashboards, please see this link: https://www.dooblo.net/stgi/order.aspx?sku=3229012

 

Step 1: Define the Operational Dashboard date range and survey

 

To define the Operational Dashboard date range and survey:

  1. Open the project and the Dashboards node:

  2. Double click the KPI’s node and set the relevant filters:

 

Step 2: Run the report

After you click the “Refresh” button you will see the different KPI’s:

 

 

 

For each KPI you can see the chart and the data. To switch to the data view, simply click on the “Data” under the relevant KPI’s:

 

 

The following KPI’s are currently available:

 

Interview Average Duration (Minutes)

 

 

This KPI shows the distribution of the interviews based on the duration in minutes. The bands are:

  • <10 minutes
  • 10-30 minutes
  • 30-60 minutes
  • 60+ minutes

 

 

Number of Interviews

 

 

This KPI shows you the amount of uploaded interviews per day. The data is based on the Upload time (see this article about the different times captured for an interview). If you hover over a date it will show you the actual number.

 

 

Interview Throughput

 


This KPI shows you the amount of uploaded interviews per hour of the day. The data is shown in a heat map where the darker shades of green represent higher interview uploads for that hour and day. The data is based on the Upload time (see this article about the different times captured for an interview).

 

 

Fieldwork Start

 


 

This KPI shows you the time of day that fieldwork started. The numbers represent the number of interviewers that did their first interview at that hour of the day on that day. For example, On July 11, 10am, 2 interviewers did their first interviews etc. The data is shown in a heat map where the darker shades of green represent higher number of interviewers that started their interviewing at that hour and day. The data is based on the Timestamp of the first interview for each interviewer on any given day. (see this article about the different times captured for an interview)

 

 

Fieldwork End

 


 

This KPI shows you the time of day that fieldwork ended. The numbers represent the number of interviewers that completed their last interview at that hour of the day on that day. For example, On July 11, 2pm, 2 interviewers completed their last interviews etc. The data is shown in a heat map where the darker shades of blue represent higher number of interviewers that completed their interviewing at that hour and day. The data is based on the Submit Time of the last interview for each interviewer on any given day. (see this article about the different times captured for an interview)

 

 

GPS Information

 

 

This KPI shows you how many interviews contained GPS location information. The GPS location information is an important quality control mechanism and there for it is highly recommended you check this KPI to make sure you are hitting above 60% of GPS information.

 

 

 

Interviewer Performance

 

 

This KPI shows is based on the average QC score interviewers receive. The KPI will show you the best performing interviewers vs the worst performing interviewers based on their average QC score received by your quality control department.

 

 

 

Visit us at: http://www.dooblo.net

How to: Use “Auto back-check”

August 20, 2013 Leave a comment

Overview

In the case you have field managers who are tasked with running back-checks on interviews of their interviewers, you can configure SurveyToGo to automatically designate interviews to be “returned to surveyor” to the field manager, so that the field manager can view them and run back checks on them. The interviews for back-checking will be chosen randomly from the pool of interviews conducted by the field manager interviewers in the last 24 hours since the last cut-off time. This allows for daily auto back-checking. In order to configure the auto back-checks:

  1. Step 1: Specify the field manager interviewers.
  2. Step 2: Configure the auto back-check settings for a survey

Step 1: Specify the field manager interviewers

In case we have a field manager user named “FM-Rick” who is in charge of 3 interviewers called “Interviewer1” , “Interviewer2” and “Interviewer3”. In order to configure the fact that FM-Rick is responsible for the 3 interviewer users:

  1. Right click on the FM-Rick user and select “QA Responsible List”:
  2. Click “Add” and locate and select the 3 interviewer users:
  3. Finally, make sure both the interviewer users and the field manager user are assigned to the relevant survey.

Step 2: Configure the auto back-check settings for a survey

Now that the interviewer-field manager relationship is established, you can configure the auto back-check settings for the survey. To do this:

  1. Right click on the survey, and select the “Auto back check information” option:
  2. Next, set the relevant settings based on the description provided

The auto back check settings screen:

 

auto-backcheck

 

Option Description
Enable auto backcheck Turns the auto back check on/off
Assign by supervisor/surveyor Checking this box sets the system to assign by supervisor. Un-checking sets the system to assign by surveyor:
By Supervisor: Means that out of the pool of the interviews performed by the interviewers assigned to the field manager, X random interviews will be randomly selected and assigned to the field manager but in no case no more than X interviews.
By Surveyor: Means that out of the pool of the interviews performed by the interviewers assigned to the field manager up to X random interviews of each interviewer will be assigned to the field manager but in no case no more than X interviews * interviewers.
Assign X interviews per supervisor/surveyor Number of interviews to assign according to the definition above
Run-at time Cut off time for the day. Recommended to leave at 12:00am as the default.
Repeat every Interval of when to run the assignments, recommended to leave at every 24 hours.
Backcheck Interviews of status Specify the status of the interviews that the system will check in order to consider for the auto-backcheck

That’s it!

Visit us at: http://www.dooblo.net

%d bloggers like this: