Flatfile plugins extend your data processing capabilities with pre-built, reusable modules for common data operations. They handle everything from file extraction and data transformation to validation and external integrations, allowing you to build sophisticated data workflows quickly.

Open Source

All Flatfile plugins are open source and available on GitHub at github.com/FlatFilers/flatfile-plugins.

Plugin Categories

🔄 Transform

Plugins for data transformation, validation, and cleaning.

📤 Extractors

Plugins for parsing and extracting data from various file formats.

📋 Schemas

Plugins for converting external schemas to Flatfile format.

📤 Export

Plugins for exporting processed data to external systems.

🔧 Core

Essential plugins for Flatfile operations and configuration.

✅ Validation

Specialized validation plugins for data quality.

🔗 Enrich

Plugins for data enrichment using external APIs.

Getting Started with Plugins

Installation

Install plugins using npm in your Flatfile project:

npm install @flatfile/plugin-record-hook @flatfile/plugin-autocast

Basic Usage

Add plugins to your listener:

import { FlatfileListener } from '@flatfile/listener'
import { recordHook } from '@flatfile/plugin-record-hook'
import { autocast } from '@flatfile/plugin-autocast'

const listener = FlatfileListener.create(async (client) => {
  // Add autocast for automatic type conversion
  client.use(autocast())
  
  // Add record hook for custom validation
  client.use(
    recordHook('customers', (record) => {
      // Custom record processing logic
      const email = record.get('email')
      if (email && !isValidEmail(email)) {
        record.addError('email', 'Invalid email format')
      }
      return record
    })
  )
})

Plugin Configuration

Most plugins accept configuration options:

import { configureSpace } from '@flatfile/plugin-space-configure'
import { dedupePlugin } from '@flatfile/plugin-dedupe'

client.use(
  configureSpace({
    workbooks: [workbookConfig],
    space: {
      name: "Customer Import",
      namespace: "customers"
    }
  })
)

client.use(
  dedupePlugin({
    sheetSlug: 'customers',
    dedupe: {
      key: 'email',
      keep: 'first',
      custom: {
        operation: 'dedupe-customers',
        label: 'Remove Duplicate Customers'
      }
    }
  })
)

Plugin Development

Creating Custom Plugins

You can create custom plugins for reusable functionality:

import { FlatfilePlugin } from '@flatfile/listener'

export function myCustomPlugin(options: any): FlatfilePlugin {
  return (client) => {
    // Plugin initialization logic
    client.filter({ job: 'sheet:myCustomOperation' }, (configure) => {
      configure.on('job:ready', async (event) => {
        // Plugin functionality
        const { jobId } = event.context
        
        try {
          await client.jobs.ack(jobId, {
            info: 'Running custom operation...'
          })
          
          // Your custom logic here
          await performCustomOperation(options)
          
          await client.jobs.complete(jobId, {
            outcome: { message: 'Custom operation completed successfully' }
          })
        } catch (error) {
          await client.jobs.fail(jobId, {
            outcome: { message: `Operation failed: ${error.message}` }
          })
        }
      })
    })
  }
}

// Usage
client.use(myCustomPlugin({ customOption: 'value' }))

Plugin Best Practices

  1. Single Responsibility: Each plugin should handle one specific functionality
  2. Configuration: Accept configuration options for flexibility
  3. Error Handling: Implement robust error handling and logging
  4. Documentation: Provide clear documentation and examples
  5. Testing: Include comprehensive tests for your plugin

Basic Data Import

// Essential plugins for most data import scenarios
client.use(autocast())
client.use(recordHook('sheet-name', validateRecord))
client.use(dedupePlugin({ sheetSlug: 'sheet-name' }))

Advanced Validation

// Comprehensive data validation
client.use(autocast())
client.use(recordHook('customers', validateCustomer))
client.use(validateEmail({ sheetSlug: 'customers', fieldKey: 'email' }))
client.use(validatePhone({ sheetSlug: 'customers', fieldKey: 'phone' }))
client.use(validateDate({ sheetSlug: 'customers', fieldKey: 'birthdate' }))

Schema Conversion

// Convert external schemas to Flatfile
client.use(convertJsonSchema({ source: 'api-schema.json' }))
client.use(convertOpenApiSchema({ source: 'swagger.yaml' }))

Data Enrichment

// Enrich data with external services
client.use(enrichGeocode({ 
  sheetSlug: 'addresses',
  addressField: 'full_address'
}))
client.use(enrichSentiment({
  sheetSlug: 'feedback', 
  textField: 'comments'
}))

Plugin Repository

All Flatfile plugins are open source and available on GitHub:

Contributing

We welcome contributions to the Flatfile plugin ecosystem:

  1. Bug Reports: Report issues on the GitHub repository
  2. Feature Requests: Suggest new plugins or enhancements
  3. Pull Requests: Contribute code improvements or new plugins
  4. Documentation: Help improve plugin documentation

Support

  • Documentation: Each plugin includes detailed documentation
  • Community: Join our Slack community for support
  • GitHub Issues: Report bugs or request features on GitHub
  • Professional Support: Contact our team for enterprise support