Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ib uses "FIFO" positioning logic - that's super anti-retail.. lame #307

Closed
goodboy opened this issue May 5, 2022 · 1 comment
Closed
Assignees
Labels
accounting prolly positioning: the accounting of "what/when (is) owned" broker-backend `brokerd`/`datad` related backend tech brokers-can-smbz features that suits should provide

Comments

@goodboy
Copy link
Contributor

goodboy commented May 5, 2022

i got lots more examples of plain old pure nonsense that goes on with this broker (and hint, issues like this are exactly why this project got started by a bunch of weirdos in the first place) but the latest is a truly exemplary inverse-hanlon's, icing-on-cake, can you be more rigged, idiotic, plain-laughable-while-despicable.

The easiest way to understand the issue in it's entirety is to read this elitetrader thread which breaks down the issue in great detail.

synopsis:

ib's stack, despite being probably the most advanced in the retail space, magically can't give you more then 7days worth of your own trade events without using their even shoddier "PLEX RePOrtz" (which you then have to parse and correlate on your own with the traditional APIs, oh and did i mention it's delayed a day...smh**2); even the worst crypto exchanges give you at least your trades history ledger (even if they don't compute pps or PnL for you 😿 ).

I'd really love to hear the explanation for why literally the most important ledger as a trader isn't available through the API over a timeframe suitable for actually calculating long running pp pnls..

What ib will give you is their horribly calculated FIFO style "average price" in PnL events and by default in TWS's (mosaic) "portfolio view". FIFO is horrible because as you exit a pp your average price changes. This is just plain idiotic: it means the trader has to manually recompute what the "magic price is" that determines whether you exit the pp at a profit or a loss (sorry to anyone who can't grok this but i have no clue how you can even compute a net-zero pnl without this). Using FIFO is mysteriously convenient for the broker (and i'm sure their larger clients who trade against most retail traders who get stuck with this accounting reporting model) yet in pretty much all the ways, makes their client's more susceptible to lose money unless they're saavy enough to compute a LIFO style pp avg price themselves. I'll say it again, i don't know how you can calculate a sane pnl with LIFO - i've experienced it first hand - the avg price will never be the price that determines the net-zero position pnl.

technicalities

  • ib's "PnL Eventz" i've now first hand experienced, do FIFO "pnl calculations" and made me take a larger loss (on a stop out) then intended since the "average price" calculated from a now scaled out of positiion was "lower" then it should have been..
  • we need to either start capturing trade logs ourselves and storing them in the db or filesystem or hacking together some "plex report" thing to get long term trade data
  • there's a slew of things to muck with in tws: (TODO)

but why aren't we reporting this issue to them?

bc honestly f#$% them. we'll hack around it ourselves just like we do everything else they have; there's no time around these parts for dealing with clearly incompetant-but-possibly-malicious, absolutely ridiculous nonsense like this.

@goodboy goodboy added the brokers-can-smbz features that suits should provide label May 5, 2022
@goodboy goodboy self-assigned this May 6, 2022
goodboy added a commit that referenced this issue May 15, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
goodboy added a commit that referenced this issue May 15, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
goodboy added a commit that referenced this issue May 15, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
goodboy added a commit that referenced this issue May 21, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
goodboy added a commit that referenced this issue May 24, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
goodboy added a commit that referenced this issue Jun 3, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
goodboy added a commit that referenced this issue Jun 5, 2022
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
@goodboy goodboy mentioned this issue Jun 7, 2022
18 tasks
@goodboy goodboy added broker-backend `brokerd`/`datad` related backend tech accounting prolly positioning: the accounting of "what/when (is) owned" labels Feb 3, 2023
@goodboy
Copy link
Contributor Author

goodboy commented Feb 3, 2023

As a follow up see our nascent sub-system for position calcs which landed as part of #336, #350, #361, #363, #365.

The last one there is mega important and eventually will be used to drive graphical support for a historical ppu (price per unit) curve overlay.

See follow up issues: #345, #384.

@goodboy goodboy closed this as completed Feb 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accounting prolly positioning: the accounting of "what/when (is) owned" broker-backend `brokerd`/`datad` related backend tech brokers-can-smbz features that suits should provide
Projects
None yet
Development

No branches or pull requests

1 participant