Quick Start Guide
SyncSage is a unified data platform that simplifies building, managing, and scaling data pipelines. It integrates:
- Data Ingestion: Connect to sources like Salesforce, Google Ads, and databases.
- Data Warehousing: Optimized Snowflake integration for scalable storage.
- Transformations: Build modular SQL models with dbt.
- Visualization: Export analytics-ready data to Power BI, Tableau, or custom D3.js dashboards.
Key Workflow:
Data Sources → Snowflake → dbt Models → Dashboards/Reports
Follow these steps to build your first pipeline in <15 minutes:
Step 1: Set Up Data Sources
- Navigate to Data Sources > Ingestion Setup.
- Select a source (e.g., Salesforce).
- Configure credentials and sync frequency.
- Click Run Initial Sync.
Step 2: Configure Snowflake Warehouse
- Go to Data Warehouse > Snowflake Integration.
- Enter Snowflake account ID, user, and password.
- Define target schemas (e.g., raw, staging).
Step 3: Build Your First dbt Model
- Open dbt Cloud IDE (Develop > Cloud IDE).
- Create a staging model.
- Click Preview to validate, then Commit.
Step 4: Visualize Data
- Connect Power BI to the analytics schema in Snowflake.
- Import the customer_360 table.
- Build a dashboard:
- Metric: Lifetime Value (LTV)
- Dimension: Customer region
Objective: Ingest raw Salesforce data, clean it, and visualize customer metrics.
Pipeline Steps
- Ingest: Sync Salesforce raw_customers to Snowflake.
- Transform:
- Clean data with stg_customers.sql
- Calculate LTV in customer_360.sql
- Visualize:
- Connect Power BI to Snowflake
- Plot LTV by region
4.1 Data Sources
- Supported Sources: Salesforce, Google Ads, PostgreSQL, REST APIs.
- Best Practices:
- Schedule hourly/daily syncs
- Use incremental loading for large datasets
4.2 Data Warehouse
- Snowflake Integration:
- Auto-scaling compute resources
- Role-based access control
- Schema Management:
- Organize tables into raw, staging, and analytics schemas
4.3 Data Transformation
- dbt Basics:
- Models: SQL files for staging, intermediate, and mart layers
- Tests: Validate uniqueness, non-null values, and relationships
Example Test:
# models/schema.yml
- name: customer_360
columns:
- name: customer_id
tests:
- unique
- not_null4.4 Data Visualization
- Power BI Integration:
- Direct query to Snowflake
- Refresh datasets on pipeline completion
- D3.js Custom Dashboards:
- Use the analytics API endpoint
1. Pipeline Design
- Separate raw, staging, and analytics layers
- Use dbt tags (e.g., daily, hourly) for job scheduling
2. Performance
- Cluster keys in Snowflake for large tables
- Limit CTEs in dbt models for readability
3. Collaboration
- Review dbt code via pull requests
- Document models with description configs
Contact Support:
- Email: [email protected]
- Live Chat: Available in the SyncSage dashboard
© 2023 SyncSage. Begin your data journey today