Custom CSV Import Architecture for Salesforce Developers

When the standard tools run out

Data Loader and the Import Wizard cover most imports. They stop being enough when the file carries complex multi-object relationships, needs custom validation logic, or has to sit behind a UI your end users can drive themselves. That is the point where you build a custom CSV import engine. The design below uses Apex, Custom Metadata Types and Batch processing to handle large datasets while respecting governor limits.

1. Defining the mapping layer

Hardcoding column-to-field mappings is a recipe for technical debt. Store the mapping configuration in Custom Metadata Types (CMT) instead, and admins can change field mappings without a code deployment.

First, define a custom metadata type named CSV_Field_Mapping__mdt with the fields Source_Column__c, Target_Field__c and Target_Object__c.

public class MappingService {
    public static Map<String, CSV_Field_Mapping__mdt> getMappings(String importType) {
        Map<String, CSV_Field_Mapping__mdt> mappingMap = new Map<String, CSV_Field_Mapping__mdt>();
        for (CSV_Field_Mapping__mdt mdt : [SELECT Source_Column__c, Target_Field__c, Target_Object__c 
                                         FROM CSV_Field_Mapping__mdt 
                                         WHERE Import_Context__c = :importType]) {
            mappingMap.put(mdt.Source_Column__c, mdt);
        }
        return mappingMap;
    }
}

2. Chunking data with Batch Apex

Load a large CSV into memory in one go and you get a Heap Size Limit Exceeded exception. Process the file in chunks. If the file arrives from a Lightning Web Component, store it in a ContentVersion record and pass the ContentDocumentId into a Database.Batchable class.

public class CSVImportBatch implements Database.Batchable<String>, Database.Stateful {
    private String content;
    
    public CSVImportBatch(String content) {
        this.content = content;
    }

    public Iterable<String> start(Database.BatchableContext bc) {
        // Split by line, ignoring header row
        return content.split('\n'); 
    }

    public void execute(Database.BatchableContext bc, List<String> lines) {
        List<SObject> recordsToInsert = new List<SObject>();
        Map<String, CSV_Field_Mapping__mdt> mappings = MappingService.getMappings('MyImport');
        
        for (String line : lines) {
            List<String> columns = line.split(',');
            // Logic to transform columns to SObject based on mappings
            recordsToInsert.add(transformRow(columns, mappings));
        }
        
        Database.insert(recordsToInsert, false); // Partial success allowed
    }
    
    public void finish(Database.BatchableContext bc) {
        // Notify user via Platform Event or Email
    }
}

3. Implementing a multi-object strategy

A single CSV row often maps to several objects, an Account and its related Contact for instance. Use the Unit of Work pattern or a parent-child processing sequence to keep the data intact. Insert the parent records first and capture the returned IDs so you can map them onto the child records inside the same transaction block.

• Parent-first: identify the parent record in the row, create it, and add it to a map keyed on a unique external ID from the CSV.
• Relational mapping: use those captured external IDs to link child records during the final DML operation.
• Validation: a try-catch inside execute lets an individual row fail without rolling back the entire batch.

4. Validation and error handling

Custom imports need real validation before anything hits the database. Implement a ValidationEngine class that evaluates business rules dynamically.

public class ValidationEngine {
    public static Boolean isValid(SObject record, Map<String, Object> data) {
        // Execute custom cross-field validation rules
        if (data.get('AnnualRevenue') < 0) return false;
        return true;
    }
}

When a row fails, capture the error in a custom Import_Error_Log__c object. Then give the user a downloadable CSV containing only the failed rows and their error messages.

5. Performance considerations

• Bulk your DML. No DML inside loops.
• Watch CPU time when the logic gets complex. If you hit the CPU limit, move logic into a Queueable Apex job or cut the batch into smaller Scope sizes.
• Run large imports through Database.Batchable so the user isn't stuck waiting for the process to complete.