A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm PLOS ONE

Clients also benefit, as internalisation reduces market impact. But when volatility rises and client flows become one-sided, market-makers must quickly pivot to external venues to hedge their risks. Whether to skew prices and wait for offsetting client flow, or hedge with other dealers in the open market, is a decision that is usually left to traders. But traders have little more than their judgment and experience to go by. This potential weakness of the analytical AS approach notwithstanding, we believe the theoretical optimality of its output approximations is not to be undervalued.

Nevertheless, in practice, deviations from the model scenarios are to be expected. Under real trading conditions, therefore, there is room for improvement upon the orders generated by the closed-form AS model and its variants. If more than 1 order_levels are chosen, multiple buy and sell limit orders will be created on both sides, with predefined price distances from each other, with the levels closest to the reservation price being set to the optimal bid and ask prices. This price distance between levels is defined as a percentage of the optimal spread calculated by the strategy. Given that optimal spreads tend to be tight, the level_distances values should be in general in tens or hundreds of percents.

P&L-to-MAP

In 2008, Avellaneda and Stoikov published a procedure to obtain bid and ask quotes for high-frequency market-making trading . The successive orders generated by this procedure maximize the expected exponential utility of the trader’s profit and loss (P&L) profile at a future time, T , for a given level of agent inventory risk aversion. Inventory management is therefore central to market making strategies , and particularly important in high-frequency algorithmic trading. In an influential paper , Avellaneda and Stoikov expounded a strategy addressing market maker inventory risk.

This strategy implements a market making strategy described in the classic paper High-frequency Trading in a Limit Order Book written by Marco Avellaneda and Sasha Stoikov. It allows users to directly adjust the risk_factor parameter GMT described in the paper. It also features an order book liquidity estimator calculating the trading intensity parameters automatically. Additionally, the strategy implements an order size adjustment algorithm and its order_amount_shape_factor parameter as described in Optimal High-Frequency Market Making. The strategy is implemented to be used either in fixed timeframes or to be ran indefinitely.

Paper statistics

At this point the trained https://www.beaxy.com/ network model had 10,000 rows of experiences and was ready to be tested out-of-sample against the baseline AS models. As we shall see shortly, the reward function is the Asymmetric dampened P&L obtained in the current 5-second time step. In contrast, the total P&L accrued so far in the day is what has been added to the agent’s state space, since it is reasonable for this value to affect the agent’s assessment of risk, and hence also how it manipulates its risk aversion as part of its ongoing actions. Together, a) and b) result in a set of 2×10d contiguous buckets of width 10−d, ranging from −1 to 1, for each of the features defined in relative terms. Approximately 80% of their values lie in the interval [−0.1, 0.1], while roughly 10% lie outside the [−1, 1] interval. Values that are very large can have a disproportionately strong influence on the statistical normalisation of all values prior to being inputted to the neural networks.

From the negative values in the Max DD columns, we see that Alpha-AS-1 had a larger Max DD (i.e., performed worse) than Gen-AS on 16 of the 30 test days. However, on 13 of those days Alpha-AS-1 achieved a better P&L-to-MAP score than Gen-AS, substantially so in many instances. Only on one day was the trend reversed, with Gen-AS performing slightly worse than Alpha-AS-1 on Max DD, but then performing better than Alpha-AS-1 on P&L-to-MAP. This is obtained from the algorithm’s P&L, discounting the losses from speculative positions. The Asymmetric dampened P&L penalizes speculative positions, as speculative profits are not added while losses are discounted.

Short-Term Market Changes and Market Making with Inventory

The avellaneda stoikov market making microstructure, which can be stated as the research on the strong trading mechanisms managed for the financial securities, has been equipped with the contributions by the books Hasbrouck and O’Hara . The question of the truncation of the interval of possible state feature values remains open, or there seems to be some misunderstanding between the authors and the reviewer. For instance, how are market prices (or actually differences to the mid-price) truncated to the interval [-1,1]? Are they scaled by some scaling parameter beforehand – and what data is this parameter estimated from ? If not, how much data is lost by only using the price differences with absolute values smaller than 1?

Finally, the best-performing model overall, with its corresponding parameter values contained in its chromosome, is retained for subsequent application to the problem at hand. In our case, it will be the AS model used as a baseline against which to compare the performance of our Alpha-AS model. The Avellaneda-Stoikov procedure underpinning the market-making actions in the models under discussion is explained in Section 2. Section 3 provides an overview of reinforcement learning and its uses in algorithmic trading. The deep reinforcement learning models (Alpha-AS-1 and Alpha-AS-2) developed to work with the Avellaneda-Stoikov algorithm are presented in detail in Section 4, together with an Avellaneda-Stoikov model (Gen-AS) without RL with parameters obtained with a genetic algorithm. Section 5 describes the experimental setup for backtests that were performed on our RL models, the Gen-AS model and two simple baselines.

Is the value function for the control problem and, moreover, the optimal controls are given by . Papers With Code is a free resource with all data licensed under CC-BY-SA. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact.

If they’ll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible — no later than 48 hours after receiving the formal acceptance.

Asset Characteristics Estimation¶

Cricket teams are ranked to indicate their supremacy over their counter peers in order to get precedence. Various authors have proposed different statistical techniques in cricketing works to evaluate teams. However, it does not work well to realize the consistency of the teams’ performance. With this aim, effective features are constructed for evaluating bowling and batting precedence of teams with others. Eventually, these features are integrated to formulate the Consistency Index Rank to rank cricket teams.

We study avellaneda stoikov market making trading strategy of a market maker with stock inventory in the presence of short-term market changes, especially changes in trading intensity of market participants and stock volatility. We employ Poisson jump processes ADA in modelling such market condition changes. We provide closed form optimal bidding and asking strategies of the market maker, and analyze the market maker’s inventory changes accordingly.