Hello, my name is Norman.


I am a quant researcher at a crypto focused hedge fund in Manhattan.

I am a PhD candidate in Canada. My research focuses on reinforcement learning. The primary objectives of my work are to improve agent performance, data efficiency, and task transferability in complex environments. My research applies to financial markets (algorithmic trading), control systems, and robotics.

Research Papers:

Policy Agnostic Successor Features: We propose a series of adjustments to the successor feature framework that allows the use of a state-transition model to dynamically create the successor features in a policy agnostic manner.

Second-Order Rewards For Successor Features: We introduce a novel formulation of the successor feature framework that models the reward as a non-linear combination of state features. The new formulation provides additional flexibility and improves performance when state features are non-perfect. A new quantity emerges that can model the environment stochasticity and can be used for guided exploration.

Noisy Importance Sampling Actor-Critic: By injecting noise into the importance sampling ratio, used to weight training samples of an on-policy algorithm, we see improved performance across several tasks in the Atari environment. Further, we show that the noise fundamentally changes how off-policy samples are weighted.

Dynamic Planning Networks: A model that learns to use a state-transition model to dynamically construct plans by optimizing reward and state novelty. This work provides evidence that it is indeed better to learn how to plan in an end-to-end manner.


I worked fulltime at Scaled Inference1 in Palo Alto, CA. My work focused on distributed systems and machine learning.

I am the author of PLE, a reinforcement learning environment for python with over 30 academic citations 750+ github stars.

Interned at Scaled Inference1 in Palo Alto, CA. My internship focused on the combination of bayesian models and deep neural networks. Additionally, I spent time improving modeling speed by porting code to run on GPUs.

Interned at Flipboard where I created a method for Image Super Resolution. While there I had to create a way to optimize model parameters and did so with bayesian optimization techniques over clusters of GPUs.

Performed research at the University of Western Ontario focusing on anomaly detection with electrical stream data using machine learning methods.


@normantasfi or email (n plus tasfi at google email)

1: ceased operations in 2019.