Understanding Salesforce Big Objects
Big Objects in Salesforce are a specialized data storage mechanism designed to hold and manage massive volumes of data (millions to billions of records) while keeping platform performance and governor limits in check. They are ideal for long-term archival, audit trails, and event logging where high volume and retention are more important than real-time relational operations.
Key characteristics of Big Objects
– Designed for extreme scale (tens to hundreds of millions, up to billions of rows).
– Schema defined in metadata (via tooling/metadata API for custom big objects).
– Immutable records: no triggers, no workflow rules, and limited DML capabilities compared to standard/custom objects.
– Primary indexing: you define a composite primary index (up to 5 fields) to optimize lookups and queries.
– Querying: supports Async SOQL (for large result sets) and limited synchronous queries; best practice is to use Async SOQL, Bulk API, or tooling APIs to extract data.
– No sharing rules, no reporting in the standard Report Builder, and no lookup/master-detail relationships like regular objects.
Common use cases
– Event logging and monitoring (audit trails, API usage logs)
– Historical data archiving (long-term retention of transactional snapshots)
– Compliance and forensic analysis where write-once/read-many patterns are acceptable
Example: Standard Big Object — EventLogFile
Salesforce provides some standard big objects out-of-the-box. One widely used example is EventLogFile, which stores event log data produced by Salesforce Event Monitoring. This big object is designed to hold vast amounts of user and system activity logs (like Login, API, URI, ApexExecution events) over time for audit and security analysis.
Typical fields in EventLogFile include:
– Id (record identifier)
– EventType (type of event, e.g., “API”, “LOGIN”)
– LogDate (timestamp for the event)
– LogFile (reference to the log file or data payload)
Async SOQL example querying EventLogFile
To query big objects efficiently, use Async SOQL. Example query payload (submitted to the Async SOQL endpoint):
SELECT Id, EventType, LogDate FROM EventLogFile WHERE EventType = 'API' AND LogDate >= 2024-01-01T00:00:00Z
Notes:
– Async SOQL returns results asynchronously and is suitable for large scans across the big object index. You submit the query via the Async SOQL REST endpoint and poll for job completion.
Best practices
– Design your primary index carefully: it drives query patterns and performance.
– Use Async SOQL for extracting large data sets and the Bulk API when moving data in/out.
– Keep in mind limitations: no triggers, limited metadata operations, and constrained reporting support.
In summary, big objects are the right choice when you need to store and query extremely large datasets in Salesforce without impacting core CRM performance. Standard big objects like EventLogFile are practical, out-of-the-box examples for event and audit log storage.