FinRL 入门指南

千年一磊 ​

Documentation guidelines

The Orleans documentation is built in Markdown. We use a few simple conventions to ensure a homogeneous style throughout the full set of documents.

These standards are being introduced. If you have issues with these guidelines then raise an issue or a Pull Request. If you find documentation that fails to meet the guidelines, then make a fix and submit a pull request. Also if you are using windows 10 you can go to the store and find free MarkDown editors FinRL 入门指南 like this



The documentation will follow US-English spelling. Desktop tools like http://markdownpad.com have spell checking features.

Paragraph structure

Each sentence should be written on a single line, and only one sentence per line. This makes merging changes easier and also helps identify verbose language.

Paragraphs in Markdown are just one or more lines of consecutive text followed by one or more blank lines.


Heading should be used to structure a document. Avoid using other emphasis features like ALLCAPS, Italics or bold to identify a new topic. Using a header is not only more consistent, but also allows linking to the header.


At the end of a page, it is helpful to link to the next logical page in the documentation. If the page is the last in a sub-section then linking back to the index page is useful.


Code formatting

Blocks of example code should be formatted with the triple back tick format followed by the language

Which will render as

Inline code should be marked with a single backtick (`).

This include references to:

  • type names e.g. Task
  • variable names e.g. game
  • namespaces e.g. Orleans.Storage.AzureTableStorage

If showing text that is an output (e.g. text file content or console output) you can either use the triple back tick without specifying a language or you can indent the content. For example:

File names and paths

When referencing a filename, directory/folder or URI then use standard italics to format. This can be done by surrounding the FinRL 入门指南 string with either with a single asterisk ( * ) or a single underscore ( _ )

  • OrleansRuntimeInterfaces.dll
  • C:\Binaries
  • ../src/Grain.cs


Markdown supports tabular data. Tables could be used to structure data so that is is easily consumable for the reader.

FinRL 入门指南
ms millisecond(s)
s second(s)
m minute(s)


When referencing another concept, the concept should be linked to. Forward FinRL 入门指南 and backward references with in a page can be linked to via the header. e.g. link back to Structure Links to other documents can either link to the page, or a sub-section/header within the page. External links should be exposed as a the full link e.g. https://github.com/dotnet/roslyn


The Orleans documentation is managed as Markdown files in a Git repository hosted on GitHub in the gh-pages branch. See the GitHub Pages documentation on how to use the gh-pages branch convention for “Project site” documents.

© 2016 Microsoft Research with help from Jekyll Bootstrap and Twitter Bootstrap

【FinRL】量化交易深度强化学习库-使用 1

千年一磊 ​

FinRL 学习记录

整体上看下来,其将金融中的相关概念构建成一个强化学习的框架(特征对应state, 买卖对应action, 收益对应return),这部分的构建将金融中很复杂的部分进行了封装,能够方便后续模型的开发的。


1. pyfolio 代码问题

FinRL 使用了 pyfolio 依赖,然而在最新版本的 pyfolio==0.9.2 版本中,在回测过程中可能会出现如下的问题:

在查阅相关的 issues,可以利用 PR#634 对这部分代码问题进行修正

2. 数据问题

FinRL 使用 Yahoo Finance API 进行获取数据,例如读取 2009-01-01 ~ 2021-01-01 的数据:

但是得到的数据却是从 2008-12-31 开始的:

另外,利用 Yahoo Finance API 在读取数据的时候,会存在有数据缺失的问题,在回测的时候就会因为缺失数据的问题报错,解决办法是对数据进行补齐(直接修改FinRL的plot.py代码):

至此, FinRL 的demo才算跑通


action 用于表示买入卖出等操作,主要包括 卖出 ,等待,买入三种操作,也支持买入卖出不同份数,即 A = \ ,其中 k 是需要指定的最大份数

reward 用来表示收益,公式为 r(s,a,s')=v'-v​ ,其中 v ​ 表示的是投资组合的总价值, r ​​ 相当于每一步决策的收益, FinRL 中还支持别的reward的计算方法,不过没有提供函数式的接口,用起来感觉相对麻烦(也可以自己写)

state 状态这块,若股票标的数量为 n ,使用的因子数量为 m ,此时的状态数为 1 + 2 * n + n * m​

  • 其中 1 表示账户余额
  • 2 * n ​ 表示,每只股票的持仓量和每只股票的股价
  • n * m 表示,表示 n 只股票的 m 个因子

所以整体上在使用的过程中,主要在于构建状态因子和reward FinRL 入门指南 的计算方式。


FinRL 中使用的 pyfolio 进行回测,这个包在回测的时候很方便,只需要输入策略的return和基准的return,就可以得到很多的分析图表,基于 FinRL ,进行了如下的实验

  • 训练标的:道琼斯指数30只成分股,日频;基准为道琼斯指数
  • 动作空间: \
  • 2009-01-01 ~ 2019-01-01 训练, 2019-01-01 ~ 2021-01-01 测试
  • 使用了15个因子
  • A2C算法

[1] Liu, Xiao-Yang, et al. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance."arXiv preprint arXiv:2011.09607(2020).

Block or report unbre

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Popular repositories

0 contributions in the last FinRL 入门指南 year

Contribution activity

August 2022


© 2022 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Load balancing in Orleans

Load balancing, in a broad sense, is one of the pillars of the Orleans runtime. Orleans runtime tries to make everything balanced, since balancing allows to maximize resource usage and avoid hotspots, which leads to better performance, as well as helps with elasticity. Load balancing in Orleans applies in multiple places. Below is FinRL 入门指南 a non-exhaustive list of places where the runtime performs balancing:

Default actor placement strategy is random - new activations are placed randomly across silos. That results in a balanced placement and prevents hotspots for most scenarios.

A more advanced ActivationCountPlacement tries to equalize the number of activations on all silos, which results in a more even distribution of activations across silos. This is especially important for elasticity.

Grain Directory service is built on top of a Distributed Hash Table, which inherently is balanced. The directory service maps grains to activations, each silo owns part of the global mapping table, and this table is globally partitioned in a balanced way across all silos. We use consistent hashing with virtual buckets for that.

Clients connect to all gateways and spread their FinRL 入门指南 requests across them, in a balanced way.

Reminder service is a distributed partitioned runtime service. The assignment of which silo is responsible to serve which reminder is balanced across all silos via consistent hashing, just like in grain directory.

Performance critical components within a silo are partitioned, and the work across them is locally balanced. That way the silo runtime can fully utilize all available CPU cores and not create in-silo bottlenecks. This applies to all local resources: allocation of work to threads, sockets, dispatch FinRL 入门指南 responsibilities, queues, etc.

StreamQueueBalance balances the responsibility of pulling events from persistence queues across silos in the cluster.

Also notice that balancing, in a broad sense, does not necessarily mean loss of locality. One can be balanced and still maintain a good locality. For example, when balancing means sharding/partitioning, you can partition responsibility for a certain logical task, while still maintaining locality within each partition. That applies both for local and distributed balancing.

Refer to FinRL 入门指南 this presentation on Balancing Techniques in Orleans for more details.

© 2016 Microsoft Research with help from Jekyll Bootstrap and Twitter Bootstrap