Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] SEC - Management Discussion & Analysis Sections #7006

Merged
merged 64 commits into from
Feb 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
b161e50
open draft
deeleeramone Jan 19, 2025
db87f1b
some table smoothing..
deeleeramone Jan 19, 2025
991ca49
fix embedded image links
deeleeramone Jan 19, 2025
2efbf3e
bullet list cleanup
deeleeramone Jan 20, 2025
819c043
some updates..coming back to it later
deeleeramone Jan 20, 2025
af7e808
Merge branch 'develop' into feature/mdna
deeleeramone Jan 21, 2025
7b2fa3e
saving here as a clear marker for progress
deeleeramone Jan 22, 2025
bc4281d
some linting
deeleeramone Jan 22, 2025
b8bfcb8
param description
deeleeramone Jan 22, 2025
c1347d4
some edge case...
deeleeramone Jan 22, 2025
12e9bd8
cch cch changes
deeleeramone Jan 23, 2025
d3b177d
Merge branch 'develop' into feature/mdna
deeleeramone Jan 24, 2025
33b8a8f
missing docstring
deeleeramone Jan 24, 2025
07ce6a9
walk back..
deeleeramone Jan 24, 2025
85994a9
f-string
deeleeramone Jan 25, 2025
cfec049
some improvements, hopefully
deeleeramone Jan 26, 2025
5d10696
some improvements, hopefully
deeleeramone Jan 26, 2025
baafea1
lint
deeleeramone Jan 26, 2025
f56b3c8
use generator instead
deeleeramone Jan 26, 2025
d28bfee
Merge branch 'develop' of https://github.com/OpenBB-finance/OpenBB in…
deeleeramone Jan 26, 2025
ca042fa
fixes some table parsing
deeleeramone Jan 26, 2025
c8e71f9
find the section in banks like JPM and C
deeleeramone Jan 27, 2025
39254f9
fixes some cell splitting behaviour
deeleeramone Jan 27, 2025
2ddf4e8
test stuff
deeleeramone Jan 27, 2025
2c883a7
lock file
deeleeramone Jan 27, 2025
a244d69
lock file
deeleeramone Jan 27, 2025
35cd7da
test things
deeleeramone Jan 27, 2025
5d122f2
remove commented out lines
deeleeramone Jan 27, 2025
821497d
startswith for bulletpoints..
deeleeramone Jan 27, 2025
b2f330c
Merge branch 'develop' into feature/mdna
deeleeramone Jan 27, 2025
e9a210a
Merge branch 'develop' into feature/mdna
deeleeramone Jan 27, 2025
0de75af
some more edges
deeleeramone Jan 28, 2025
e5d58cb
O
deeleeramone Jan 28, 2025
059589d
try not to be tricked into starting at the TOC
deeleeramone Jan 28, 2025
a5d9eff
font difference causing start not to be found.
deeleeramone Jan 28, 2025
9219436
fix wfc that calls the section something else entirely
deeleeramone Jan 28, 2025
c635a16
fix target header not being all on the same line
deeleeramone Jan 28, 2025
f4efff2
fix target section missing Item 2 or Item 7 in text
deeleeramone Jan 28, 2025
fdd6af1
fix for BKNG
deeleeramone Jan 28, 2025
38f9d80
fix different wording order in target section title
deeleeramone Jan 28, 2025
ff0ab44
fix some error with BX
deeleeramone Jan 28, 2025
ae63516
fix some nested table bullet points
deeleeramone Jan 28, 2025
9a157d9
fix out of range
deeleeramone Jan 28, 2025
ce742be
Merge branch 'develop' into feature/mdna
jmaslek Jan 28, 2025
597e403
fix to find intel's section title
deeleeramone Jan 28, 2025
baaddd3
Merge branch 'feature/mdna' of https://github.com/OpenBB-finance/Open…
deeleeramone Jan 28, 2025
0db77ca
fix APH issue finding start
deeleeramone Jan 28, 2025
b342c8f
some edge cases
deeleeramone Jan 29, 2025
16381ab
Help out a lil
jmaslek Jan 29, 2025
7a082af
add 'strategy' param to select parser
deeleeramone Jan 30, 2025
c52f677
some patches
deeleeramone Jan 30, 2025
427bece
allow entering CIK for symbol
deeleeramone Jan 30, 2025
c4795c7
update parsing bool on failed attempt
deeleeramone Jan 30, 2025
d58b202
integration params
deeleeramone Jan 30, 2025
4a3189b
add warning when trafilatura fails, and don't insert cell dividers wh…
deeleeramone Jan 30, 2025
c376e3f
fix section headers
deeleeramone Jan 30, 2025
20ee5dc
format fixes
deeleeramone Jan 30, 2025
1080e40
more formatting
deeleeramone Jan 30, 2025
497613c
linter
deeleeramone Jan 30, 2025
0e09000
Merge branch 'develop' into feature/mdna
deeleeramone Jan 30, 2025
2a6b5ad
Merge branch 'develop' of https://github.com/OpenBB-finance/OpenBB in…
deeleeramone Jan 31, 2025
b7d6ef6
add widget config for markdown
deeleeramone Jan 31, 2025
d16b543
calendar period as Q1-Q4 enum
deeleeramone Feb 1, 2025
ea87fba
test param
deeleeramone Feb 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""Management Discussion & Analysis Standard Model."""

from datetime import date as dateType
from typing import Literal, Optional

from openbb_core.provider.abstract.data import Data
from openbb_core.provider.abstract.query_params import QueryParams
from openbb_core.provider.utils.descriptions import (
DATA_DESCRIPTIONS,
QUERY_DESCRIPTIONS,
)
from pydantic import Field, field_validator


class ManagementDiscussionAnalysisQueryParams(QueryParams):
"""Management Discussion & Analysis Query Parameters."""

symbol: str = Field(description=QUERY_DESCRIPTIONS.get("symbol", ""))
calendar_year: Optional[int] = Field(
default=None,
description="Calendar year of the report. By default, is the current year."
+ " If the calendar period is not provided, but the calendar year is, it will return the annual report.",
)
calendar_period: Optional[Literal["Q1", "Q2", "Q3", "Q4"]] = Field(
default=None,
description="Calendar period of the report. By default, is the most recent report available for the symbol."
+ " If no calendar year and no calendar period are provided, it will return the most recent report.",
)

@field_validator("symbol", mode="before", check_fields=False)
@classmethod
def to_upper(cls, v: str):
"""Convert field to uppercase."""
return v.upper()


class ManagementDiscussionAnalysisData(Data):
"""Management Discussion & Analysis Data."""

symbol: str = Field(description=DATA_DESCRIPTIONS.get("symbol", ""))
calendar_year: int = Field(description="The calendar year of the report.")
calendar_period: int = Field(description="The calendar period of the report.")
period_ending: Optional[dateType] = Field(
description="The end date of the reporting period.", default=None
)
content: str = Field(
description="The content of the management discussion and analysis."
)
30 changes: 30 additions & 0 deletions openbb_platform/extensions/equity/integration/test_equity_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -2189,3 +2189,33 @@ def test_equity_ownership_government_trades(params, headers):
result = requests.get(url, headers=headers, timeout=10)
assert isinstance(result, requests.Response)
assert result.status_code == 200


@parametrize(
"params",
[
(
{
"symbol": "AAPL",
"calendar_year": 2024,
"calendar_period": "Q2",
"wrap_length": 120,
"include_tables": False,
"use_cache": True,
"raw_html": False,
"strategy": "trafilatura",
"provider": "sec",
}
),
],
)
@pytest.mark.integration
def test_equity_fundamental_management_discussion_analysis(params, headers):
"""Test the equity fundamental management discussion analysis endpoint."""
params = {p: v for p, v in params.items() if v}

query_str = get_querystring(params, [])
url = f"http://0.0.0.0:8000/api/v1/equity/fundamental/management_discussion_analysis?{query_str}"
result = requests.get(url, headers=headers, timeout=10)
assert isinstance(result, requests.Response)
assert result.status_code == 200
Original file line number Diff line number Diff line change
Expand Up @@ -2049,3 +2049,30 @@ def test_equity_ownership_government_trades(params, obb):
assert result
assert isinstance(result, OBBject)
assert len(result.results) > 0


@parametrize(
"params",
[
(
{
"symbol": "AAPL",
"calendar_year": 2024,
"calendar_period": "Q2",
"wrap_length": 120,
"include_tables": False,
"use_cache": True,
"raw_html": False,
"strategy": "trafilatura",
"provider": "sec",
}
),
],
)
@pytest.mark.integration
def test_equity_fundamental_management_discussion_analysis(params, obb):
"""Test the equity fundamental management discussion analysis endpoint."""
result = obb.equity.fundamental.management_discussion_analysis(**params)
assert result
assert isinstance(result, OBBject)
assert len(result.results.content) > 0
Original file line number Diff line number Diff line change
Expand Up @@ -477,3 +477,58 @@ async def trailing_dividend_yield(
) -> OBBject:
"""Get the 1 year trailing dividend yield for a given company over time."""
return await OBBject.from_query(Query(**locals()))


@router.command(
model="ManagementDiscussionAnalysis",
examples=[
APIEx(parameters={"symbol": "AAPL", "provider": "sec"}),
APIEx(
description="Get the Management Discussion & Analysis section by calendar year and period.",
parameters={
"symbol": "AAPL",
"calendar_year": 2020,
"calendar_period": "Q4",
"provider": "sec",
},
),
APIEx(
description="Setting 'include_tables' to True will attempt to extract all tables in valid Markdown.",
parameters={
"symbol": "AAPL",
"calendar_year": 2020,
"calendar_period": "Q4",
"provider": "sec",
"include_tables": True,
},
),
APIEx(
description="Setting 'raw_html' to True will bypass extraction and return the raw HTML file, as is."
+ " Use this for custom parsing or to access the entire HTML filing.",
parameters={
"symbol": "AAPL",
"calendar_year": 2020,
"calendar_period": "Q4",
"provider": "sec",
"raw_html": True,
},
),
],
openapi_extra={
"widget_config": {
"type": "markdown",
"data": {"dataKey": "results.content", "columnsDefs": []},
"staleTime": 86400000,
"refetchInterval": 86400000,
"source": "SEC",
}
},
)
async def management_discussion_analysis(
cc: CommandContext,
provider_choices: ProviderChoices,
standard_params: StandardParams,
extra_params: ExtraParams,
) -> OBBject:
"""Get the Management Discussion & Analysis section from the financial statements for a given company."""
return await OBBject.from_query(Query(**locals()))
154 changes: 154 additions & 0 deletions openbb_platform/openbb/assets/reference.json
Original file line number Diff line number Diff line change
Expand Up @@ -26646,6 +26646,160 @@
},
"model": "TrailingDividendYield"
},
"/equity/fundamental/management_discussion_analysis": {
"deprecated": {
"flag": null,
"message": null
},
"description": "Get the Management Discussion & Analysis section from the financial statements for a given company.",
"examples": "\nExamples\n--------\n\n```python\nfrom openbb import obb\nobb.equity.fundamental.management_discussion_analysis(symbol='AAPL', provider='sec')\n# Get the Management Discussion & Analysis section by calendar year and period.\nobb.equity.fundamental.management_discussion_analysis(symbol='AAPL', calendar_year=2020, calendar_period=4, provider='sec')\n# Setting 'include_tables' to True will attempt to extract all tables in valid Markdown.\nobb.equity.fundamental.management_discussion_analysis(symbol='AAPL', calendar_year=2020, calendar_period=4, provider='sec', include_tables=True)\n# Setting 'raw_html' to True will bypass extraction and return the raw HTML file, as is. Use this for custom parsing or to access the entire HTML filing.\nobb.equity.fundamental.management_discussion_analysis(symbol='AAPL', calendar_year=2020, calendar_period=4, provider='sec', raw_html=True)\n```\n\n",
"parameters": {
"standard": [
{
"name": "symbol",
"type": "str",
"description": "Symbol to get data for.",
"default": "",
"optional": false,
"choices": null
},
{
"name": "calendar_year",
"type": "int",
"description": "Calendar year of the report. By default, is the current year. If the calendar period is not provided, but the calendar year is, it will return the annual report.",
"default": null,
"optional": true,
"choices": null
},
{
"name": "calendar_period",
"type": "int",
"description": "Calendar period of the report. By default, is the most recent report available for the symbol. If no calendar year and no calendar period are provided, it will return the most recent report.",
"default": null,
"optional": true,
"choices": null
}
],
"sec": [
{
"name": "wrap_length",
"type": "int",
"description": "The length to wrap the extracted text, excluding tables. Default is 120.",
"default": 120,
"optional": true,
"choices": null
},
{
"name": "include_tables",
"type": "bool",
"description": "Return tables formatted as markdown in the text. Default is False. Tables may reveal 'missing' content, but will likely need some level of manual cleaning, post-request, to display properly. In some cases, tables may not be recoverable due to the nature of the document.",
"default": false,
"optional": true,
"choices": null
},
{
"name": "use_cache",
"type": "bool",
"description": "When True, the file will be cached for use later. Default is True.",
"default": true,
"optional": true,
"choices": null
},
{
"name": "raw_html",
"type": "bool",
"description": "When True, the raw HTML content of the entire filing will be returned. Default is False. Use this option to parse the document manually.",
"default": false,
"optional": true,
"choices": null
}
]
},
"returns": {
"OBBject": [
{
"name": "results",
"type": "List[ManagementDiscussionAnalysis]",
"description": "Serializable results."
},
{
"name": "provider",
"type": "Optional[Literal['sec']]",
"description": "Provider name."
},
{
"name": "warnings",
"type": "Optional[List[Warning_]]",
"description": "List of warnings."
},
{
"name": "chart",
"type": "Optional[Chart]",
"description": "Chart object."
},
{
"name": "extra",
"type": "Dict[str, Any]",
"description": "Extra info."
}
]
},
"data": {
"standard": [
{
"name": "symbol",
"type": "str",
"description": "Symbol representing the entity requested in the data.",
"default": "",
"optional": false,
"choices": null
},
{
"name": "calendar_year",
"type": "int",
"description": "The calendar year of the report.",
"default": "",
"optional": false,
"choices": null
},
{
"name": "calendar_period",
"type": "int",
"description": "The calendar period of the report.",
"default": "",
"optional": false,
"choices": null
},
{
"name": "period_ending",
"type": "date",
"description": "The end date of the reporting period.",
"default": null,
"optional": true,
"choices": null
},
{
"name": "content",
"type": "str",
"description": "The content of the management discussion and analysis.",
"default": "",
"optional": false,
"choices": null
}
],
"sec": [
{
"name": "url",
"type": "str",
"description": "The URL of the filing from which the data was extracted.",
"default": "",
"optional": false,
"choices": null
}
]
},
"model": "ManagementDiscussionAnalysis"
},
"/equity/ownership/major_holders": {
"deprecated": {
"flag": null,
Expand Down
Loading