Senior Data Operations Engineer

RESPONSIBILITIES

Database Design & Architecture

• Design, implement, and maintain optimal database schemas for relational (MariaDB) and NoSQL (MongoDB) databases.
• Participate in data modeling efforts to support analytics in Databricks.

Performance Monitoring & Tuning

• Monitor and tune performance across all platforms to ensure optimal performance.
• Use profiling tools (e.g., query plans, system logs) to identify and resolve bottlenecks.

Security & Compliance

• Implement access controls, encryption, and database hardening techniques.
• Manage user roles and privileges across MariaDB, MongoDB, and Databricks.
• Ensure compliance with data governance policies (e.g., GDPR, HIPAA).

Backup & Recovery

• Implement and maintain backup/recovery solutions for all database platforms.
• Periodically test restore procedures for business continuity.

Data Integration & ETL Support

• Support and optimize ETL pipelines between MongoDB, MariaDB, and Databricks.
• Work with data engineers to integrate data sources for analytics.

Monitoring & Incident Response

• Set up and monitor database alerts.
• Troubleshoot incidents, resolve outages, and perform root cause analysis.

MariaDB-Specific Responsibilities

• Administer MariaDB instances (standalone, replication, Galera Cluster).
• Optimize SQL queries and indexing strategies.
• Maintain stored procedures, functions, and triggers.
• Manage schema migrations and upgrades with minimal downtime.
• Ensure ACID compliance and transaction management.

MongoDB-Specific Responsibilities

• Manage replica sets and sharded clusters.
• Perform capacity planning for large document collections.
• Tune document models and access patterns for performance.
• Set up and monitor MongoDB Ops Manager / Atlas (if used).
• Automate backup and archival strategies for NoSQL data.

Databricks-Specific Responsibilities

• Manage Databricks workspace permissions and clusters.
• Collaborate with data engineers to optimize Spark jobs and Delta Lake usage.
• Ensure proper data ingestion, storage, and transformation in Databricks.
• Support CI/CD deployment of notebooks and jobs.
• Integrate Databricks with external data sources (MariaDB, MongoDB, S3, ADLS)

Collaboration & Documentation

• Collaborate with developers, data scientists, and DevOps engineers.
• Maintain up-to-date documentation on data architecture, procedures, and standards.
• Provide training or onboarding support for other teams on database tools.

REQUIREMENTS

• MariaDB Tools: mysqldump, mysqladmin, Percona Toolkit
• MongoDB Tools: mongodump, mongotop, mongoexport, Atlas UI
• Databricks Tools: Jobs UI, Databricks CLI, REST API, SQL Analytics
• Scripting: Bash, Python, PowerShell
• Monitoring: Prometheus, Grafana, CloudWatch, DataDog
• Version Control & CI/CD: Git, Jenkins, Terraform (for infrastructure-as-code)
• Preferred CLoud provider: AWS