About the Role
Zignal Labs’ real-time intelligence technology helps the world’s largest organizations protect their people, places, and position. Analyzing billions of data points in real time, Zignal's AI-powered platform accelerates mission-critical decision making by empowering leaders with contextual situational awareness of the information environment.
Fully remote, with Silicon Valley roots and team members in over 20 states, Zignal serves customers around the world. Learn more at zignallabs.com.
As the Principal Data Engineer on the Platform team at Zignal Labs, you will get to use your Scala and Java experience to build a best-in-class distributed data and analytics infrastructure by leveraging open source technologies such as Apache Spark, Apache Storm, and Elasticsearch. We use social media, news, blogs and other media sources to empower our users with key insights based on real-time analysis.
In this role, you will have the opportunity to:
Solve complex real-time data collection & analysis problems with cutting edge technical solutions
Iterate on our high-performance and scalable platform for massive data collection, real-time analytics, NLP, machine learning, and backend data services
Build high-performance, scalable, real-time, server-side technologies
Write scalable code with extensive test coverage, working in a professional software engineering environment with source control, dev/stage/production release cycles, continuous integration, and deployment
Work closely with product management, design, quality assurance and operations teams to understand our customers’ needs and effectively translate them to technical specifications
Lead projects from translating product requirements into architecture to production
Tech Stack:
Scala, Java, Python
Apache Spark, Spark Streaming, Databricks/Delta Lake, Apache Storm, Elasticsearch, Apache Nifi
Kafka, MongoDB, Redis
AWS
In order to be successful in this role, you will need:
Bachelor's degree (or higher) in Computer Science, Engineering, or similar and/or relevant work experience
Experience providing technical leadership at the enterprise level for the design of information technology systems
Crafted and implemented operational data stores, as well as data lakes in production environments
Ability to analyze, diagnose and resolve complex architectural problems using industry standard engineering principles
Design and build data ingestion pipelines and ETL processing, including stream processing, while factoring in performance and cost
Identify and solve issues concerning data management to improve data quality
Clean, prepare and optimize data for ingestion and consumption
Experience solving performance problems with Lucene based search solutions like Elasticsearch or Solr
9+ years experience in server-side/back-end full cycle product development in a production environment
4+ years developing with Apache Spark, including Structured Streaming. Experience with Databricks is a big plus
Knowledge of Scala or Java with exposure to or interest in Scala
Leads and mentors other team members
Provides partners with coaching and feedback in order to build effective teams
Provides effective support to cross-functional teams
$170,000 - $190,000 a year
The salary range is applicable to the general US and may be adjusted based on geographic location. Compensation decisions depend on the circumstances of each case, including skill sets, experience, certifications, and business and organizational needs.
Why join Zignal Labs?
- Competitive salary based on the work you do
- Flexible time off – work with your manager to take the time you need
- Excellent medical, dental, and vision coverage
- Paid parental leave plan
- Professional development and growth programs
- A tight knit, collaborative, and transparent environment to help you succeed