MOHIT KUMAR

Technical Consultant and Trainer

"I am fascinated with Computer Science, mathematics, Universe and stuff. I like to understand things and the way they work under-the-hood. Occasionally, I like to explain the things that I understand using the first principle's approach. More formally I am a Researcher, Trainer, and a design consultant on the design of Artificial Intelligence(deep learning)based systems. Microarchitecture based optimizations are my specialization, more specifically P5(Intel) to Skylake(Intel). On the vector side, SIMD, GPUs(Nvidia). My micro-architecture knowledge enable me to see the complete stack. Close to 20 years in total experience, last 5 years I have been working on Optimizing Tensorflow and Models on Tensorflow on GPUs/CPUs. A general example of micro-optimization on Haswell microarchitecture."

Artificial Intelligence Research (DEEP Breath)

I feel that the barrier to entry for Deep Learning is very steep. Consider Natural Language Processing as an example. Neural Machine Translation, for example, uses concepts like LSTM, Bidirectional LSTM, Multi Layered LSTMs, Attention, etc. Neither one of them is easy to understand by itself, imagine the plight of a student when these concepts are strung together for a Neural Machine Translation or Google's BERT based systems. I have seen Neural Machine Translation Based systems grossly underperform and, it was simply because most of the hyperparameters were not understood at all.
DEEP-Breathe is a complete and pure python implementation of most complex models, especially but not limited to Neural Machine Translator.

It has 2 distinct uses: 1. Gaining an intimate understanding of concepts that can be applied in practice. I build my Training Slides (RNN, LSTM, NMT, Attention) and Blog to complement the code at DEEP-Breathe. I also picturize the code so it is much easier to reason about in most complex scenarios. DEEP-Breathe open-sourced with MIT Licence. It is already being used in many organizations internally.

2. It is used as a custom optimization tool if the complete architecture of the model is known ahead-of-time (Proprietary).

Concurrency

My main interests are techniques for designing, implementing, and reasoning about multiprocessor algorithms, in particular concurrent data structures for multicore machines and the mathematical foundations of the computation models that govern their behavior.

My research these days is directed at the use of randomness and combinatorial techniques in concurrent algorithm and data-
structure design.

My work: I specialize in core JDK concurrent libraries included in version 5,6,7,8. This would include not just the theoretical understanding but also how the data structures works internally. My work often is to map real world perfomances to the underlying data structures. Derive a theoretical analysis based on the CPU (and sometimes GPU) cycles consumed. But this would not be possible without a step-by-step(transitive) in-memory layout of the structure. I use the same transitive in-memory layout for training purposes (Figure-1). Link above demonstrates that in detail (Slides 3-5 are code points marked and slides 6-24 are object structure creation as it happens in the memory). I also specialize in Non JDK libraries like the Highly-Scalable-Library, Guava, and others.

JVM Tunings and Optimizations

Designing, Tuning and, Optimization of JVM based applications. Most often tuning is not only a software job. It requires one to know the entire stack right down to the hardware and the tools with which one can pinpoint the real issue. Beside the usual profilers, the tools I specialize in are Perf, Systemtap, Dtrace, Solaris studio analyzer, JMH, JCStress etc.

Most often tuning is not only a software job and majority of the java CPU profilers have little idea as what is happening beyond the JVM.
So using the right profiler with minimal overhead is the key. PMCs ( Performance Monitoring Counters) are special CPU registers that can record the entire trace of a call. Perf and system tap are tools that make this extremely easy and then generate a flame graph(figure-2) to inspect the calls that are bottlenecks.

On-cpu flame graph of ArrayblockingQueue in action. Off-cpu flame graph of ArrayblockingQueue in action. Flame graph of FileInputStream in action.

With solaris studio analyzer one can associate a cache miss, a resource stall on the CPU pipeline to java source code. The tool can measure almost any hardware counter and associate that with java source code, byte code or, generated assembly from the JVM (figure-3). The link shows CPU profiling session in a bit more detail. A blog article to explain it

Ultra Low Latency design and Architecture

With an intimate knowledge of the hardware specially the CPU(x86 family mostly) and the GPU, I specialize in the design and architecture of ultra low latency software.

My Work:

Traditional models of concurrent programming have been around for some time. They work well and have matured quite a bit over the last couple of years. However, every once in a while, there are low latency and/or high throughput requirements that cannot be met by traditional models of concurrency and application design.

How about handling 700-800 million operations/second per core of a system on the JVM platform/C++. When designing such systems one has to throw away the traditional models of application design and think different. I specialize in the approaches that can make this possible. The approaches are hardware friendly and a re-look at data from a hardware perspective.

It requires logical understanding of how modern hardware works. It also requires the knowledge of tools that can help track down a particular stall and possibly the reason behind it. This may provide pointers for a redesign if required. It is about designing very specialized data structures that fully exploits the underlying architecture of the CPU (and/or GPU), cache, memory, disk, filesystem, and the network. Here is an example of super optimized Ultra Low Latency messaging framework.

Read more

My main interests are techniques for designing, implementing, and reasoning about multiprocessor algorithms, in particular concurrent data structures for multicore machines and the mathematical foundations of the computation models that govern their behavior.

My research these days is directed at the use of randomness and combinatorial techniques in concurrent algorithm and data-
structure design.

Read more

Read more

With an intimate knowledge of the hardware specially the CPU(x86 family mostly) and the GPU, I specialize in the design and architecture of ultra low latency software.

Read more

Read more

My main interests are techniques for designing, implementing, and reasoning about multiprocessor algorithms, in particular concurrent data structures for multicore machines and the mathematical foundations of the computation models that govern their behavior.

My research these days is directed at the use of randomness and combinatorial techniques in concurrent algorithm and data- structure design.

Read more

Read more

With an intimate knowledge of the hardware specially the CPU(x86 family mostly) and the GPU, I specialize in the design and architecture of ultra low latency software.

Read more

My research these days is directed at the use of randomness and combinatorial techniques in concurrent algorithm and data-
structure design.