Async Soql Guide
Async Soql Guide
@salesforcedocs
Last updated: December 18, 2017
© Copyright 2000–2017 salesforce.com, inc. All rights reserved. Salesforce is a registered trademark of salesforce.com, inc.,
as are other names and marks. Other marks appearing herein may be trademarks of their respective owners.
CONTENTS
Async SOQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
ASYNC SOQL
Async SOQL is a method for running SOQL queries when you can’t wait for the results in real time.
EDITIONS
These queries are run in the background over Salesforce entity data, standard objects, custom
objects, and big objects. It provides a convenient way to query large amounts of data stored in Available in: Enterprise,
Salesforce. Performance, Unlimited,
Async SOQL is implemented as a RESTful API that enables you to run queries in the familiar syntax and Developer Editions
of SOQL. Because of its asynchronous operation, you can subset, join, and create more complex
queries and not be subject to timeout limits. This situation is ideal when you have millions or billions
of records and need more performant processing than is possible using synchronous SOQL. The results of each query are deposited into
an object you specify, which can be a standard object, custom object, or big object.
You can schedule multiple queries and monitor their completion status. The limit for Async SOQL queries is one concurrent query at a
time.
1
Async SOQL
For example, let’s say that you want to analyze the years and years of opportunity history collected by Salesforce. The results could help
you identify which current and future opportunities are more likely to close and give you a better picture of your forecast. But because
the opportunity history data is stored with all the field history data across the application, the volume of data is too large to query directly.
That’s where Async SOQL comes in! You can use it to write a query that extracts a smaller, representative subset of the data that you’re
interested. You can store this working dataset in a custom object and use it in reports, dashboards, or any other Force.com feature.
2
Async SOQL
With big objects, you can now bring a much finer level of detail into your applications using data that you already have. For example,
every interaction an individual has with your marketing campaign is stored as data that you can use, but it’s unwieldy in its raw form.
With Async SOQL, you can aggregate that data by campaign and day, allowing you to extract the relevant details of the full dataset into
a smaller, usable dataset. As in the previous example, the smaller working set can live in a custom object and be used in your reports
and dashboards.
3
RUNNING ASYNC SOQL QUERIES
Learn how to run Async SOQL queries on your objects and check on the status of your query using the Chatter REST API.
operation String Specify whether the query is an insert or upsert. If Optional 39.0
the record doesn’t exist, an upsert behaves like an
insert.
targetObject String A standard object, custom object, external object, Required 35.0
or big object into which to insert the results of the
query.
targetFieldMap Map<String, Defines how to map the fields in the query result Required 35.0
String> to the fields in the target object.
4
Running Async SOQL Queries
targetValueMap Map<String, Defines how to map static strings to fields in the Optional 37.0
String> target object. Any field or alias can be used as the
TargetValueMap value in the SELECT clause
of a query.
You can map the special value, $JOB_ID, to a field
in the target object. The target field must be a
lookup to the Background Operation standard
object. In this case, the ID of the Background
Operation object representing the Async SOQL
query is inserted. If the target field is a text field, it
must be at least 15–18 characters long.
You can also include any field or alias in the SELECT
clause of the TargetValueMap. They can be
combined together to concatenate a value to be
used.
targetExternalIdField String The ID of the target sObject. Required for upsert Optional 39.0
operations.
This simple Async SOQL example queries SourceObject__c, a source custom object, and directs the result to TargetObject__c,
another custom object. You can easily map the fields in the source object to the fields of the target object in which you want to write
the results.
Example URI
https://yourInstance.salesforce.com/services/data/v38.0/async-queries/
"operation": "insert",
"targetObject": "TargetObject__c",
"targetFieldMap": {"firstField__c":"firstFieldTarget__c",
"secondField__c":"secondFieldTarget__c"
},
"targetValueMap": {"$JOB_ID":"BackgroundOperationLookup__c",
"Copy fields from source to
target":"BackgroundOperationDescription__c"
}
}
5
Running Async SOQL Queries
The response of an Async SOQL query includes the elements of the initial POST request.
Response body for POST
message String A text message that provides information Big, 37.0 37.0
regarding the query, such as an error
message if the query failed.
operation String Specify whether the query is an insert or Big, 39.0 .39.0
upsert. If the record doesn’t exist, an upsert
behaves like an insert.
query String Specifies the parameters for the SOQL query Big, 35.0 35.0
you want to execute.
targetExternalIdField String The ID of the target sObject. Required for Big, 39.0 39.0
upsert operations.
6
Running Async SOQL Queries
targetValueMap Map<String, String> Defines how to map static strings to fields Big, 37.0 37.0
in the target object. Any field or alias can be
used as the TargetValueMap value in
the SELECT clause of a query.
You can map the special value, $JOB_ID, to
a field in the target object. The target field
must be a lookup to the Background
Operation standard object. In this case, the
ID of the Background Operation object
representing the Async SOQL query is
inserted. If the target field is a text field, it
must be at least 15–18 characters long.
You can also include any field or alias in the
SELECT clause of the TargetValueMap.
They can be combined together to
concatenate a value to be used.
targetObject String A standard object, custom object, external Big, 35.0 35.0
object, or big object into which to insert the
results of the query.
7
Running Async SOQL Queries
"message": "",
"status": "New",
"targetObject": "TargetObject__c",
"targetFieldMap": {"firstField__c":"firstFieldTarget__c",
"secondField__c":"secondFieldTarget__c"
},
"targetValueMap": {"$JOB_ID":"BackgroundOperationLookup__c",
"Copy fields from source to
target":"BackgroundOperationDescription__c"
}
}
The response is similar to the initial POST response but with updated status and message fields to reflect the status.
Example GET response body
{
"jobId": "08PD000000000001",
"message": "",
"query": "SELECT firstField__c, secondField__c FROM SourceObject__c",
"status": "Complete",
"targetObject": "TargetObject__c",
"targetFieldMap": {"firstField__c":"firstFieldTarget__c",
"secondField__c":"secondFieldTarget__c" }
}
You can get status information for all queries with the following HTTP GET request.
https://yourInstance.salesforce.com/services/data/v38.0/async-queries/
{
"asyncQueries" : [ {
"jobId" : "08PD00000000002",
"message" : "",
"query" : "SELECT String__c FROM test__b",
8
Running Async SOQL Queries
"status" : "Running",
"targetFieldMap" : {
"String__c" : "String__c"
},
"targetObject" : "test__b",
"targetValueMap" : { }
}, {
"jobId": "08PD000000000001",
"message": "Complete",
"query": "SELECT firstField__c, secondField__c FROM SourceObject__c",
"status": "Complete",
"targetObject": "TargetObject__c",
"targetFieldMap": {"firstField__c":"firstFieldTarget__c",
"secondField__c":"secondFieldTarget__c" }
}
}
Canceling a Query
You can cancel a query using an HTTP DELETE request by specifying its jobId.
https://yourInstance.salesforce.com/services/data/v38.0/async-queries/jobId
9
ASYNC SOQL USE CASES
https://yourInstance—api.salesforce.com/services/data/v38.0/async-queries/
"targetObject": "Rider_Reduced__b",
"targetFieldMap": {"End_Location_Lat__c":"End_Lat__c",
"End_Location_Lon__c":"End_Long__c",
"Start_Location_Lat__c": "Start_Lat__c",
"Start_Location_Lon__c": "Start_Long__c",
"End_Time__c": "End_Time__c",
"Start_Time__c": "Start_Time__c",
"Car_Type__c": "Car_Type__c",
"Rider__r.FirstName": "First_Name__c",
"Rider__r.LastName": "Last_Name__c",
"Rider__r.Email": "Rider_Email__c"
}
}
"message": "",
10
Async SOQL Use Cases
"status": "New",
"targetFieldMap": {"End_Location_Lat__c":"End_Lat__c",
"End_Location_Lon__c":"End_Long__c",
"Start_Location_Lat__c": "Start_Lat__c",
"Start_Location_Lon__c": "Start_Long__c",
"End_Time__c": "End_Time__c",
"Start_Time__c": "Start_Time__c",
"Car_Type__c": "Car_Type__c",
"Rider__r.FirstName": "First_Name__c",
"Rider__r.LastName": "Last_Name__c",
"Rider__r.Email": "Rider_Email__c"
},
"targetObject": "Rider_Reduced__b"
}
https://yourInstance.salesforce.com/services/data/v38.0/async-queries/
"targetObject": "ArchivedAccounts__b",
11
Async SOQL Use Cases
"Id": "Id__c",
"NewValue": "NewValue__c",
"OldValue": "OldValue__c"
}
}
Note: All number fields returned from a SOQL query of archived objects are in standard notation, not scientific notation, as in the
number fields in the entity history of standard objects.
Event Monitoring
Login Forensics and Real-Time Events, currently in pilot, enable you to track who is accessing confidential and sensitive data in your
Salesforce org. You can view information about individual events or track trends in events to swiftly identify unusual behavior and
safeguard your company’s data. This feature is useful for compliance with regulatory and audit requirements.
Note: These features are available to select customers through a pilot program. To be nominated to join this pilot program,
contact salesforce.com.
In the current pilot, you can monitor data accessed through API calls, which covers many common scenarios because more than 50%
of SOQL queries occur using the SOAP, REST, or Bulk APIs. Key information about each query, such as the Username, UserId, UserAgent,
and SourceIP, is stored in the ApiEvent object. You can then run SOQL queries on this object to find out details of user activity in your
org.
For example, let’s say you want to know everyone who viewed the contact record of your company’s CEO, Jane Doe. The key to this
query is the CEO’s contact record ID: 003D000000QYVZ5. (You can also query the ID using SOQL: SELECT Id FROM Contact
WHERE Name = 'Jane Doe'). You can use the following Async SOQL query to determine all users who saw their contact
information, including when, how, and where they saw it.
Example URI
https://yourInstance—api.salesforce.com/services/data/v38.0/async-queries/
"targetObject": "QueryEvents__c",
12
Async SOQL Use Cases
"targetFieldMap": {"Soql":"QueryString__c","SourceIp":"IPAddress__c",
"Username":"User__c", "EventTime":"EventTime__c",
"UserAgent":"UserAgent__c"
}
}
"message": "",
"status": "Complete",
"targetObject": "QueryEvents__c",
"targetFieldMap": {"Soql":"QueryString__c","SourceIp":"IPAddress__c",
"Username":"User__c", "EventTime":"EventTime__c", "UserAgent":"UserAgent__c"
}
}
Note: All number fields returned from a SOQL query of archived objects are in standard notation, not scientific notation, as in the
number fields in the entity history of standard objects.
If you ask this question on a repeated basis for audit purposes, you can automate the query using a cURL script.
curl -H "Content-Type: application/json" -X POST -d
'{"query": "SELECT Soql, SourceIp, UserAgent, Username, EventTime FROM ApiEvent WHERE
RecordInfo Like'%003D000000QYVZ5%'","targetObject": "QueryEvents__c",
"targetFieldMap": {"Soql":"QueryString__c", "SourceIp":"IPAddress__c", "Username":"User__c",
"EventTime":"EventTime__c",UserAgent}}'
"https://yourInstance.salesforce.com/services/data/v35.0/async-queries" -H
"Authorization: Bearer 00D30000000V88A!ARYAQCZOCeABy29c3dNxRVtv433znH15gLWhLOUv7DVu.
uAGFhW9WMtGXCul6q.4xVQymfh4Cjxw4APbazT8bnIfxlRvUjDg"
Another event monitoring use case is to identify all users who accessed a sensitive field, such as Social Security Number or Email. For
example, you can use the following Async SOQL query to determine the users who saw social security numbers and the records in which
those numbers were exposed.
Example URI
https://yourInstance—api.salesforce.com/services/data/v38.0/async-queries/
"targetObject": "QueryEvents__c",
13
Async SOQL Use Cases
}
}
"message": "",
"status": "Complete",
"targetObject": "QueryEvents__c"
}
14
SUPPORTED SOQL COMMANDS
Async SOQL supports a subset of commands in the SOQL language. The subset includes the most common commands that are relevant
to key use cases.
WHERE
Comparison operators
=, !=, <, <=, >, >=, LIKE
Logical operators
AND, OR
Date formats
YYYY-MM-DD, YYYY-MM-DDThh:mm:ss-hh:mm
Example
SELECT AnnualRevenue
FROM Account
WHERE NumberOfEmployees > 1000 AND ShippingState = ‘CA’
Date Functions
Date functions in Async SOQL queries allow you to group or filter data by time periods, such as day or hour.
Method Details
DAY_ONLY() Returns a date representing the day portion of a dateTime field.
HOUR_IN_DAY() Returns a number representing the hour in the day for a dateTime field.
Example
SELECT DAY_ONLY(date__c), HOUR_IN_DAY(date__c), COUNT(Id)
FROM FieldHistoryArchive
GROUP BY DAY_ONLY(date__c), HOUR_IN_DAY(date__c)
15
Supported SOQL Commands
Aggregate Functions
AVG(field), COUNT(field), COUNT_DISTINCT(field), SUM(field), MIN(field), MAX(field)
Example
SELECT COUNT(field)
FROM FieldHistoryArchive
HAVING
Use this command to filter results from aggregate functions.
Example
SELECT LeadSource, COUNT(Name)
FROM Lead
GROUP BY LeadSource HAVING COUNT (Name) > 100
GROUP BY
Use this option to avoid iterating through individual query results. Specify a group of records instead of processing many individual
records.
Example
SELECT COUNT(Id) count, CreatedById createdBy
FROM FieldHistoryArchive
GROUP BY CreatedById
Relationship Queries
Single-level child-to-parent relationships are supported using dot notation. Use these queries with the SELECT, WHERE, and GROUP BY
clauses.
Example
SELECT Account.ShippingState s, COUNT(Id) c
FROM Contact
GROUP BY Account.ShippingState
16
Supported SOQL Commands
17
INDEX
18