Description:
We’re seeking a talented and experienced Senior Engineer for Network Observability to join our Network Observability team. In this role, you will be a key player in designing, developing, and maintaining the monitoring, telemetry, and observability systems that keep CoreWeave’s GPU cloud network operating reliably and at scale. You’ll focus on building solutions that provide real-time insights into network performance, ensuring that issues are detected proactively and resolved quickly.
Your mission? To empower CoreWeave’s network with advanced observability: robust metrics, powerful analytics, and automated alerting—so well-tuned that any anomalies become clear before they ever impact our customers.
- Develop, optimize, and maintain network observability platforms. Use your skills in Python and Golang to create and automate collectors, exporters, and dashboards that provide deep visibility into network health and performance.
- Collaborate with Network Engineering and Platform teams to ingest and unify logs, metrics, and events from a variety of platforms (Arista EOS, NVIDIA Cumulus Linux, Nokia SR OS, SR Linux, etc.) into a single observability pipeline.
- Design and implement scalable telemetry solutions using protocols like gNMI, SNMP, and streaming analytics. Ensure advanced alerting and anomaly detection with tools such as Prometheus, Grafana, and Alertmanager.
- Work closely with network developers, site reliability engineers, and security teams to integrate observability solutions across the broader infrastructure. Participate in design discussions, RFCs, and architectural decisions.
- Join a rotating on-call schedule to troubleshoot and resolve observability-related issues. Provide timely support to operations teams, quickly isolating and fixing problems when they arise.
- Guide junior team members, share best practices, and foster a culture of continuous learning and improvement within the observability domain.
Investing in our people is one of our top priorities, and we value candidates who can bring their diversified experiences to our teams. Here are some qualities we’ve found compatible with our team. We'd love to talk about whether this aligns with your experience and interests and what you’re excited to work on next.
Who You Are
Minimum Qualifications
- Deep familiarity with Prometheus, Grafana, Alertmanager, gNMI, and SNMP. Experience writing or extending custom metric collectors/exporters is a plus.
- Experience as a Network Engineer, SRE, Software Developer, or Systems Administrator in large-scale environments. A track record of building and operating robust telemetry and monitoring solutions is a plus.
- Passion for automating tasks and processes. You find satisfaction in creating workflows that handle repetitive tasks and reduce human error to near zero.
- Comfortable containerizing solutions in Kubernetes, designing, building, and deploying container-based workloads efficiently.
- Proficient with Python, Go, and Bash, plus familiarity with configuration management and templating tools (e.g., Ansible, Jinja2). .
- Strong knowledge of Linux systems and IP networking concepts, with hands-on experience in routing, switching, and network troubleshooting.
- Practical knowledge with a variety of platforms, including Arista EOS, NVIDIA Cumulus Linux, Nokia SR OS, and SR Linux.
- Collaborative, humble, and always ready to help others while staying open to learning from more senior colleagues.
Preferred Qualifications
- College Education: Bachelor’s degree in Computer Science or a related field.
- Machine Learning for Anomaly Detection: Hands-on experience applying ML techniques or tools (e.g., TensorFlow, scikit-learn) to proactively detect performance or security anomalies in network traffic.
- Network Certifications: Certifications like CCNA, CCNP, or similar.
- Advanced Metrics & Analytics: Hands-on experience with data pipelines, event correlation, or anomaly detection in large-scale environments.
- Distributed Tracing: Familiarity with OpenTelemetry, Jaeger, or Zipkin for end-to-end tracing across microservices and network components.