Big Data Engineer (Hadoop)
G-Research is a leading quantitative research and technology company. By using the latest scientific techniques, we produce world-beating predictive research and build advanced technology to analyse the world’s data.
We are developing the next generation of infrastructure that can effortlessly scale and flex. This capability allows us to adapt to the demanding needs of the business to process and analyse large and varied data sets at speed.
You will be critical in driving the expansion of Hadoop and associated technologies and their reach within G-Research. You will use these technologies to build systems of high quality, with a focus on scalability, resiliency, security and automation.
Who are we looking for?
The ideal candidate will have experience with:
- Designing, running and troubleshooting Hadoop clusters
- Batch and streaming job frameworks (such as Spark, Storm)
- NoSQL databases (HBase, Cassandra, MongoDB)
- Middlewares and messaging systems (e.g. Kafka, RabbitMQ, FTL, Ultra Messaging)
- Linux OS core principles, performance and tuning
- Scripting (e.g. Bash, Python, Perl)
- Automation via the use of configuration management (such as Puppet or Chef) and orchestration tools (such as Ansible).
The following would be advantageous but not necessary:
- Service discovery (e.g. Zookeeper, Consul, etc)
- Data collection and querying (such as Flume, Sqoop, Hive)
- Time series databases (such as InfluxDB, OpenTSDB, or Prometheus)
- Kerberos, SSL certificates
- Other scalable distributed systems, such as Splunk
- Solid knowledge of basic network protocols (e.g. IP, UDP, TCP), OS network stacks. Multicast is a plus
- Database administration and querying (SQL)
- Cluster managers (e.g. Docker, Apache Mesos, Kubernetes)
Why should you apply?
- Highly competitive compensation plus annual discretionary bonus
- Informal dress code and excellent work/life balance
- Comprehensive healthcare and life assurance
- 25 days holiday
- 9% company pension contributions
- Cycle-to-work scheme
- Subsidised gym membership
- Monthly company events
- Central London office close to 5 stations and 6 tube lines