https://blog.turboawesome.win/series/hft-foundations/ Recent content in Hft-Foundations on Bits, Trades & Systems Hugo en-us Thu, 12 Nov 2015 16:03:00 +0000 https://blog.turboawesome.win/2015/11/five-years-in-high-frequency-trading-what-i-actually-learned/ Thu, 12 Nov 2015 16:03:00 +0000 https://blog.turboawesome.win/2015/11/five-years-in-high-frequency-trading-what-i-actually-learned/ Looking back at five years of low-latency systems work: what the environment teaches you that you can't learn elsewhere, and what habits it instils that don't always transfer cleanly to other domains. https://blog.turboawesome.win/2015/08/scala-akka-actors-for-trading-workflows-promises-and-pitfalls/ Wed, 19 Aug 2015 11:28:00 +0000 https://blog.turboawesome.win/2015/08/scala-akka-actors-for-trading-workflows-promises-and-pitfalls/ We evaluated Akka actors for a workflow orchestration layer in the trading system. The actor model's isolation and message-passing properties were genuinely useful — and its failure modes were genuinely painful. https://blog.turboawesome.win/2015/04/memory-mapped-files-in-java-chronicle-and-the-art-of-zero-copy-i/o/ Wed, 15 Apr 2015 09:53:00 +0000 https://blog.turboawesome.win/2015/04/memory-mapped-files-in-java-chronicle-and-the-art-of-zero-copy-i/o/ Memory-mapped files let you treat disk storage as if it's RAM — the OS handles the mapping and caching transparently. Chronicle Queue uses this to give you a persistent ordered log with throughput approaching raw memory bandwidth. https://blog.turboawesome.win/2015/01/chronicle-queue-vs-kafka-choosing-a-persistent-journal-at-nanosecond-scale/ Wed, 21 Jan 2015 10:44:00 +0000 https://blog.turboawesome.win/2015/01/chronicle-queue-vs-kafka-choosing-a-persistent-journal-at-nanosecond-scale/ Both Chronicle Queue and Kafka provide persistent, ordered message logs. Their performance profiles, operational models, and use cases are almost completely different. https://blog.turboawesome.win/2014/10/benchmarking-without-lying-jmh-coordinated-omission-and-honest-numbers/ Wed, 29 Oct 2014 10:14:00 +0000 https://blog.turboawesome.win/2014/10/benchmarking-without-lying-jmh-coordinated-omission-and-honest-numbers/ Most latency benchmarks are wrong in the same ways. JMH solves the JVM measurement problems; coordinated omission is the harder issue that lives above it. https://blog.turboawesome.win/2014/09/aeron-reliable-udp-multicast-for-market-data-distribution/ Wed, 17 Sep 2014 13:28:00 +0000 https://blog.turboawesome.win/2014/09/aeron-reliable-udp-multicast-for-market-data-distribution/ Aeron is a message transport library built around reliable UDP and memory-mapped files, designed for the sub-microsecond latency and high throughput that market data distribution demands. Here's what it looks like in practice. https://blog.turboawesome.win/2014/05/busy-spinning-vs-blocking-thread-strategies-for-ultra-low-latency/ Wed, 14 May 2014 10:31:00 +0000 https://blog.turboawesome.win/2014/05/busy-spinning-vs-blocking-thread-strategies-for-ultra-low-latency/ Blocking a thread hands control to the OS scheduler and costs you microseconds on wake-up. Busy spinning wastes a CPU core. The right choice depends on your latency target and hardware budget. https://blog.turboawesome.win/2014/01/scala-on-the-hot-path-where-the-abstraction-cost-goes/ Wed, 08 Jan 2014 09:44:00 +0000 https://blog.turboawesome.win/2014/01/scala-on-the-hot-path-where-the-abstraction-cost-goes/ Scala's abstractions are elegant — but on the JVM, elegance has a bytecode cost. Understanding where Scala generates boxing, closures, and virtual dispatch tells you which patterns are safe in hot paths and which aren't. https://blog.turboawesome.win/2013/09/java-chronicle-off-heap-persistence-without-serialisation-overhead/ Thu, 05 Sep 2013 14:22:00 +0000 https://blog.turboawesome.win/2013/09/java-chronicle-off-heap-persistence-without-serialisation-overhead/ Chronicle Map and Chronicle Queue use memory-mapped files to give you persistent storage that doesn't touch the GC heap. Here's how we used it for trade journaling at microsecond latency. https://blog.turboawesome.win/2013/05/comparing-arrayblockingqueue-to-the-disruptor-numbers-dont-lie/ Wed, 22 May 2013 13:39:00 +0000 https://blog.turboawesome.win/2013/05/comparing-arrayblockingqueue-to-the-disruptor-numbers-dont-lie/ A direct benchmark comparison between Java's ArrayBlockingQueue and the LMAX Disruptor at various producer/consumer configurations. What the numbers show and why. https://blog.turboawesome.win/2013/04/disruptor-deep-dive-memory-layout-cache-lines-and-false-sharing/ Tue, 09 Apr 2013 11:27:00 +0000 https://blog.turboawesome.win/2013/04/disruptor-deep-dive-memory-layout-cache-lines-and-false-sharing/ Why the Disruptor is fast is inseparable from how it's laid out in memory. A ground-up look at the cache line strategy that makes the numbers possible. https://blog.turboawesome.win/2013/02/the-lmax-disruptor-how-a-ring-buffer-changed-my-mental-model-of-queues/ Thu, 28 Feb 2013 09:55:00 +0000 https://blog.turboawesome.win/2013/02/the-lmax-disruptor-how-a-ring-buffer-changed-my-mental-model-of-queues/ The Disruptor pattern replaced our internal queue infrastructure and cut p99 latency in half. Here's the mental model shift required to understand why it works. https://blog.turboawesome.win/2013/01/introduction-to-lock-free-programming-in-java/ Thu, 17 Jan 2013 10:55:00 +0000 https://blog.turboawesome.win/2013/01/introduction-to-lock-free-programming-in-java/ Lock-free programming replaces mutex-based synchronisation with atomic CPU instructions. The performance gains are real; the correctness requirements are unforgiving. https://blog.turboawesome.win/2012/12/mechanical-sympathy-writing-java-that-respects-the-hardware/ Tue, 04 Dec 2012 10:42:00 +0000 https://blog.turboawesome.win/2012/12/mechanical-sympathy-writing-java-that-respects-the-hardware/ Martin Thompson's term 'mechanical sympathy' reframed how I think about software performance. Here's what it means in practice when writing Java for latency-sensitive systems. https://blog.turboawesome.win/2012/09/latency-vs-throughput-the-false-dichotomy-i-learned-the-hard-way/ Tue, 25 Sep 2012 10:33:00 +0000 https://blog.turboawesome.win/2012/09/latency-vs-throughput-the-false-dichotomy-i-learned-the-hard-way/ Optimising for throughput and optimising for latency are often portrayed as opposites. In practice they pull in different directions but are not mutually exclusive — and confusing them leads to systems that are bad at both.