Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

earnings_date failing (KeyError: 'Earnings Date') #1932

Closed
yunhongz opened this issue May 14, 2024 · 45 comments
Closed

earnings_date failing (KeyError: 'Earnings Date') #1932

yunhongz opened this issue May 14, 2024 · 45 comments

Comments

@yunhongz
Copy link

Describe bug

The earnings_dates record is not available

Simple code that reproduces your problem

import yfinance as yf
msft = yf.Ticker('MSFT')
msft.earnings_dates

Debug log

image

Bad data proof

No response

yfinance version

yfinance 0.2.38

Python version

python3.10

Operating system

ubuntu

@ValueRaider
Copy link
Collaborator

ValueRaider commented May 14, 2024

Yahoo has changed table structure

                                  Event Name  Earnings Call Time  EPS Estimate  Reported EPS  Surprise(%)
0                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
1                      Q1 2024 Earnings Call                 TAS           NaN           NaN          NaN
2                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
3   at Bank of America Healthcare Conference                 TAS           NaN           NaN          NaN
4                Annual Shareholders Meeting                 TAS          1.95          1.59      -0.1846
5                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
6                   Q3 2023 Earnings Release  After Market Close           NaN           NaN          NaN
7                   Q4 2023 Earnings Release  After Market Close           NaN           NaN          NaN
8   at Bank of America Healthcare Conference                 TAS         -0.74           NaN          NaN
9                   Q1 2024 Earnings Release   Time Not Supplied           NaN           NaN          NaN
10               Annual Shareholders Meeting   Time Not Supplied           NaN           NaN          NaN
11                  Q4 2023 Earnings Release  After Market Close           NaN           NaN          NaN

Event name is nice addition, but losing dates a problem. Hoping for a way to get old structure.

Probably Yahoo has broken their redesign rollout, because their Find earnings for symbol search doesn't work: https://finance.yahoo.com/calendar/earnings?symbol=MSFT

@ValueRaider ValueRaider pinned this issue May 14, 2024
@hjlgood
Copy link
Contributor

hjlgood commented May 15, 2024

Looks like yahoo fixed their problem. yfinance works fine now. I suggest you close this issue.

@ValueRaider
Copy link
Collaborator

I suspect it will return, I've seen this alternative table randomly rarely over last few weeks. Yahoo is up to something...

@michaelmoopenn
Copy link

For some tickers, it works and some others it doesn't. At this time, ENB.TO works, TSLA still gives the issue.

@yunhongz
Copy link
Author

It returned to normal, interesting!

@ValueRaider ValueRaider unpinned this issue May 29, 2024
@hjlgood
Copy link
Contributor

hjlgood commented Jun 8, 2024

Looks like Yahoo finance's earnings tab blew up again.

@ValueRaider ValueRaider pinned this issue Jun 8, 2024
@EvanYu0816
Copy link

This works fine now, but it's also pulling up non-quarterly earnings.

Depending on how Yahoo Finance plans to move forward with the earnings table, can we please add an option in the future to filter out the non-quarterly earnings?

Right now, we can tell which ones are quarterly by the Company column:
https://finance.yahoo.com/calendar/earnings?symbol=MCD

However, this column is omitted in the get_earnings function.

@ValueRaider
Copy link
Collaborator

Keeping column Company is fine, but filtering rows should be optional in case algorithm has bug.

@tomatofrankie
Copy link

Having the issue starting from this week.

@ValueRaider ValueRaider unpinned this issue Jul 21, 2024
@kasaquan
Copy link

As of today, some tickers work perfectly fine, some tickers give the 'Earnings Date' KeyError.

@xuewei4d
Copy link

when I tried to get the earnings of AI. https://finance.yahoo.com/calendar/earnings/?symbol=AI, yahoo may need some time to run js. I also tried selenium. It works but much slower.

@ValueRaider ValueRaider changed the title earnings_dates not available KeyError: 'Earnings Date' Nov 12, 2024
@ValueRaider
Copy link
Collaborator

ValueRaider commented Nov 12, 2024

I encountered this recently. I think what happens is, when Yahoo is overloaded they use a second data provider which returns a completely different table structure.

@ValueRaider ValueRaider changed the title KeyError: 'Earnings Date' earnings_date failing (KeyError: 'Earnings Date') Nov 20, 2024
@NayrusLove
Copy link

Did the table structure blow up again. I'm getting an error

@ValueRaider ValueRaider pinned this issue Nov 20, 2024
@spitrespectable
Copy link

Did the table structure blow up again. I'm getting an error

Yes, it is failing for me for multiple (50+) tickers (starting yesterday/last night)

@rvimalakempegowda
Copy link

rvimalakempegowda commented Nov 21, 2024

Even I'm facing the same issue.
Enter the company's ticker symbol: AMD

stacktrace Traceback (most recent call last): File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\base.py", line 3805, in get_loc return self._engine.get_loc(casted_key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc File "pandas\\_libs\\hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas\\_libs\\hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'Earnings Date'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\Revanth\Desktop\Final Project\Earnigns_date\earnings.py", line 57, in
earnings_dates = get_earnings_dates(ticker)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\Desktop\Final Project\Earnigns_date\earnings.py", line 28, in get_earnings_dates
earnings_dates = stock.get_earnings_dates(limit=10)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\utils.py", line 104, in wrapper
result = func(args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\yfinance\base.py", line 634, in get_earnings_dates
tzinfo = dates[cn].str.extract('([AP]M[a-zA-Z]
)$')
~~~~~^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\frame.py", line 4090, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Revanth\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\base.py", line 3812, in get_loc
raise KeyError(key) from err
KeyError: 'Earnings Date'

@mrizzben
Copy link

Did the table structure blow up again. I'm getting an error

Yeah, I'm having issues with earnings_dates too since a couple days ago

@JavierCalzadaEspuny
Copy link

Same, does anyone know what's going on this time?

@michaelmoopenn
Copy link

The data return by python request doesn't execute the filter that you would have on a browser:
https://finance.yahoo.com/calendar/earnings?symbol=GOOGL&offset=0&size=12

[   Symbol                       Company                                         Event Name  Earnings Call Time EPS Estimate Reported EPS Surprise(%)
0   FORTY      Formula Systems 1985 Ltd                           Q3 2024 Earnings Release  Before Market Open            -            -           -
1   WLMSQ                    NOS4-1 Inc  Q4 2023 Williams Industrial Services Group Inc...   Time Not Supplied            -            -           -
2   VTIAF                    Vita 34 AG                           Q3 2024 Earnings Release                 TAS            -            -           -
3    ALPP         Alpine 4 Holdings Inc                           Q2 2024 Earnings Release  After Market Close            -            -           -
4    ALPP         Alpine 4 Holdings Inc                           Q1 2024 Earnings Release  After Market Close            -            -           -
..    ...                           ...                                                ...                 ...          ...          ...         ...

This is why "Earnings Date" is not there. The javascript needs to be executed.

@Bristopher
Copy link

Bristopher commented Nov 22, 2024

The data return by python request doesn't execute the filter that you would have on a browser: https://finance.yahoo.com/calendar/earnings?symbol=GOOGL&offset=0&size=12

[   Symbol                       Company                                         Event Name  Earnings Call Time EPS Estimate Reported EPS Surprise(%)
0   FORTY      Formula Systems 1985 Ltd                           Q3 2024 Earnings Release  Before Market Open            -            -           -
1   WLMSQ                    NOS4-1 Inc  Q4 2023 Williams Industrial Services Group Inc...   Time Not Supplied            -            -           -
2   VTIAF                    Vita 34 AG                           Q3 2024 Earnings Release                 TAS            -            -           -
3    ALPP         Alpine 4 Holdings Inc                           Q2 2024 Earnings Release  After Market Close            -            -           -
4    ALPP         Alpine 4 Holdings Inc                           Q1 2024 Earnings Release  After Market Close            -            -           -
..    ...                           ...                                                ...                 ...          ...          ...         ...

This is why "Earnings Date" is not there. The javascript needs to be executed.

Ah I'm also having this issue, tried using selenium/pyppeteer and it just recently broke like two days ago and it also gives that generic response (US User)

@ValueRaider
Copy link
Collaborator

ValueRaider commented Nov 22, 2024

This is why "Earnings Date" is not there. The javascript needs to be executed.

@michaelmoopenn The table you quoted is the new format, it is missing "Earnings Date" column. Your link shows new format. Can you post a screenshot of successfully fetching the old format?

@michaelmoopenn
Copy link

michaelmoopenn commented Nov 22, 2024

This is why "Earnings Date" is not there. The javascript needs to be executed.

@michaelmoopenn The table you quoted is the new format, it is missing "Earnings Date" column. Your link shows new format. Can you post a screenshot of successfully fetching the old format?

If you go on the link with your browser you should see this:
image
Which is the correct old format. However if you click on "Clear" which clear the filter you get what python request get:
image

The issue is that python request don't get the same view as with your browser even though there is the filter attributes in the get parameters.

There is some post-processing javascript that is not executed.
Even with Selenium it doesn't get the right filtered table.

@ValueRaider
Copy link
Collaborator

If you go on the link with your browser you should see this:

I don't see that. It's not adblock etc, I turned that off. Maybe it's geo-specific - I'm Europe.

@michaelmoopenn
Copy link

I am North America. This one looks challenging to fix!

@JavierCalzadaEspuny
Copy link

So, is this gonna stay permanently returning error for EU users ?

@JimMcHale
Copy link

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

@michaelmoopenn
Copy link

If you go on the link with your browser you should see this:

I don't see that. It's not adblock etc, I turned that off. Maybe it's geo-specific - I'm Europe.

The filter works only when you are logged in.

@ValueRaider
Copy link
Collaborator

The filter works only when you are logged in.

You're right. So probably yfinance needs to support logging in.

@dislazy
Copy link

dislazy commented Nov 25, 2024

I just tried logging into yahoo and it showed me the normal earnings dates, so that might be a good solution

@szapiro
Copy link

szapiro commented Nov 25, 2024

I just tried logging into yahoo and it showed me the normal earnings dates, so that might be a good solution

True, when logged in to Yahoo is works fine.
I also found that replacing:
dates = stock.earnings_date
with:
date = stock.calendar['Earnings Date'][0]
worked for me.

@davidlarcher
Copy link

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

Hello Jim, When I try it, I don't get all the previous earnings (I just get the last 4 ones and tried for many different tickers ...)
Did I miss something ?
Screenshot from 2024-11-29 17-41-52

@JimMcHale
Copy link

Hi David,
for a stock like 'F' when I do:
self.cTicker.earnings_dates()
I get: builtins.KeyError: 'Earnings Date'
(this had worked a few weeks ago without issue; so it seemed yahoo changed something)

whereas if I used earnings_history() I get:
self.cTicker.get_earnings_history()
epsEstimate epsActual epsDifference surprisePercent
2023-12-31 0.55761 0.54 -0.02 -0.0316
2024-03-31 0.53056 0.55 0.02 0.0366
2024-06-30 0.57366 0.57 -0.00 -0.0064
2024-09-30 0.57103 0.60 0.03 0.0507

I do see (as shown by others above) that if am not logged into yahoo then when I go to the earnings page
https://finance.yahoo.com/calendar/earnings?symbol=F&offset=0&size=12
I do not get just the earnings for F but a list of all the earnings for "today"..

when I am logged in to yahoo that url shows me the upcoming earnings as well as several of the past ones..

Is it that I need to somehow login to yahoo thru yfin?
tks
-jim

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

Hello Jim, When I try it, I don't get all the previous earnings (I just get the last 4 ones and tried for many different tickers ...) Did I miss something ?

@AlphaGit
Copy link

For anyone struggling to extract the earnings from yfinance right now, I have this little snippet that got me out of trouble. It's not a drop-in replacement for yfinance, but those, like me, that needed to get the historical earnings to another platform, this might save you some trouble while this gets fixed.

  1. Navigate to this link: https://finance.yahoo.com/calendar/earnings?symbol=GOOG&offset=0&size=100
  2. Sign in to Yahoo with your own account (top right corner), you now should see the last 100 earnings for Google. Either change the filter manually to select another symbol or change the symbol parameter in the URL
  3. Open your browser's console and run this snippet, which will download the visible table as a CSV:
const earnings_json = $$('#fin-cal-table tbody tr')
	.map(tr => ({
		symbol: $('td:nth-child(1)', tr).innerText,
		date: new Date($('td:nth-child(3) span:nth-child(1)', tr).innerText.substring(0, 12)),
	}))
	.filter(e => new Date() > e.date)
	.map(e => ({
		...e,
		date: e.date.toISOString().split('T')[0],
	}))

const rows = [
	["symbol", "date"],
	...earnings_json.map(e => [e.symbol, e.date])
]

let csvContent = "data:text/csv;charset=utf-8," + rows.map(r => r.join(",")).join("\n")

var encodedUri = encodeURI(csvContent)
var link = document.createElement("a")
link.setAttribute("href", encodedUri)
link.setAttribute("download", `earnings_${earnings_json[0].symbol}.csv`)
link.click()

You can remove the filter() line if you don't want to filter future earning dates. Or, change the direction of that comparison if you only want future earnings (for me, I only want past ones).

Hope this helps.

@davidlarcher
Copy link

Thank you for your answers, login in does change the data displayed as well for me (I am in EU). I guess that indeed a login method in the class YfData along with the cookies handling would be helpful if possible ?

@jon085
Copy link

jon085 commented Dec 3, 2024

I just tried logging into yahoo and it showed me the normal earnings dates, so that might be a good solution

True, when logged in to Yahoo is works fine. I also found that replacing: dates = stock.earnings_date with: date = stock.calendar['Earnings Date'][0] worked for me.

@szapiro - your sample somehow doesn't work for example on AAPL.
I get the following dates:
'Earnings Date': [
datetime.date(2025, 1, 30), datetime.date(2025, 2, 3)
]

There are 4 dates in 2025 listed and only 1 matches the above.
https://finance.yahoo.com/calendar/earnings?symbol=AAPL&offset=0&size=12

@AlphaGit
Copy link

AlphaGit commented Dec 4, 2024

Bad news: Yahoo is changing their layout again.

Good news: there's a new URL that displays all earnings dates, hopefully a quick change for a fix.

Old URL: https://finance.yahoo.com/calendar/earnings?symbol=GOOG&offset=0&size=100
New URL: https://finance.yahoo.com/calendar/earnings?day=2024-12-04&symbol=GOOG&offset=0&size=100

(Adding the day parameter gets a different layout, that displays the whole list of earning dates.)

@Alex-Naxitus
Copy link

Alex-Naxitus commented Dec 4, 2024

Like @AlphaGit said, Yahoo changed their API route and their table formatting.
To fix this bug, make these two changes in base.py :

  1. Load the datetime package in base.py :
    from datetime import date

  2. Replace these lines (576 to 661) in base.py:

     logger = utils.get_yf_logger()
    
     page_size = min(limit, 100)  # YF caps at 100, don't go higher
     page_offset = 0
     dates = None
     while True:
         url = f"{_ROOT_URL_}/calendar/earnings?day={date.today()}&symbol={self.ticker}&offset={page_offset}&size={page_size}"
         data = self._data.cache_get(url=url, proxy=proxy).text
    
         if "Will be right back" in data:
             raise RuntimeError("*** YAHOO! FINANCE IS CURRENTLY DOWN! ***\n"
                                "Our engineers are working quickly to resolve "
                                "the issue. Thank you for your patience.")
    
         try:
             data = pd.read_html(StringIO(data))[0]
         except ValueError:
             if page_offset == 0:
                 # Should not fail on first page
                 if "Showing Earnings for:" in data:
                     # Actually YF was successful, problem is company doesn't have earnings history
                     dates = utils.empty_earnings_dates_df()
             break
         if dates is None:
             dates = data
         else:
             dates = pd.concat([dates, data], axis=0)
    
         page_offset += page_size
         # got less data then we asked for or already fetched all we requested, no need to fetch more pages
         if len(data) < page_size or len(dates) >= limit:
             dates = dates.iloc[:limit]
             break
         else:
             # do not fetch more than needed next time
             page_size = min(limit - len(dates), page_size)
    
     if dates is None or dates.shape[0] == 0:
         _exception = YFEarningsDateMissing(self.ticker)
         err_msg = str(_exception)
         logger.error(f'{self.ticker}: {err_msg}')
         return None
     dates = dates.reset_index(drop=True)
    
     # Drop redundant columns
     dates = dates.drop(["Symbol", "Company"], axis=1)
    
     # Rename surprise column
     dates=dates.rename(columns={"Surpise (%)": "Surprise(%)"})
    
     # Convert types
     for cn in ["EPS Estimate", "Reported EPS", "Surprise(%)"]:
         dates.loc[dates[cn] == '-', cn] = float("nan")
         dates[cn] = dates[cn].astype(float)
    
     # Convert % to range 0->1:
     dates["Surprise(%)"] *= 0.01
    
     # Parse earnings date string
     cn = "Earnings Date"
     dates[cn] = dates[cn].astype(str)
    
     # - remove "at"
     dates[cn] = dates[cn].replace(' at', ',', regex=True)
    
     # - remove EDT/EST 
     dates[cn] = dates[cn].replace(' E[DS]T', '', regex=True)
    
     # - remove AM/PM and timezone from date string
     tzinfo = dates[cn].str.extract('([AP]M[a-zA-Z]*)$')
     dates[cn] = dates[cn].replace(' [AP]M[a-zA-Z]*$', '', regex=True)
    
     # - split AM/PM from timezone
     tzinfo = tzinfo[0].str.extract('([AP]M)([a-zA-Z]*)', expand=True)
     tzinfo.columns = ["AM/PM", "TZ"]
    
     # - combine and parse
     dates[cn] = dates[cn] + ' ' + tzinfo["AM/PM"]
     dates[cn] = pd.to_datetime(dates[cn], format="%B %d, %Y, %I %p")
     # - instead of attempting decoding of ambiguous timezone abbreviation, just use 'info':
     self._quote.proxy = proxy or self.proxy
     tz = self._get_ticker_tz(proxy=proxy, timeout=30)
     dates[cn] = dates[cn].dt.tz_localize(tz)
    
     dates = dates.set_index("Earnings Date")
    
     self._earnings_dates[limit] = dates
    
     return dates
    

I'm not a contributor, so if you are one and you're reading this, feel free to commit this change and link this issue with your commit. As a side note, this fix only supports Eastern U.S. Time (EDT/EST) earnings dates for now.
Edit : Renamed "Surprise (%)" into "Surprise(%)" and to make it consistent with the previous output columns naming conventions.

@michaelmoopenn
Copy link

Great for the new URL!
Quick notice, after looking closer to the earnings dates, the dates are either 1 day behind or 1 day ahead. Data is inacurate.
I guess that they are still working on it...

@Alex-Naxitus
Copy link

Great for the new URL! Quick notice, after looking closer to the earnings dates, the dates are either 1 day behind or 1 day ahead. Data is inacurate. I guess that they are still working on it...

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

@AlphaGit
Copy link

AlphaGit commented Dec 7, 2024

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

@Alex-Naxitus
Copy link

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

There are systematic errors across many different tickers on the Yahoo Finance website. Yahoo Finance devs must have changed something somewhere and didn't realize they broke anything. They should have done regression tests and they didn't.

In the meantime, until they fix the error, I would suggest finding a source of data outside of Yahoo Finance (if you really need the data).

@michaelmoopenn
Copy link

On released earnings they looks to display: accurate earnings date - 1 day.

@ForeverZyh
Copy link

ForeverZyh commented Dec 11, 2024

On released earnings they looks to display: accurate earnings date - 1 day.

Do you think it is because all date is previously in UTC format at 12am? And then that's the reason why it is -1 day and also that's why they have difference for EST and EDT?

@Bristopher
Copy link

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

What's your reference for dates, I can't find a good second source to refence that includes the times.

@Alex-Naxitus
Copy link

Alex-Naxitus commented Dec 20, 2024

@Alex-Naxitus

I contacted the Yahoo Finance Data support team ; they didn't seem to be aware there were any errors in their data. They said they'll check it out, I'll keeep you posted.

I have a few examples across different symbols that we had to manually fix. Feel free to reach out if you think it would help in figuring out the source of the errors. (It's far from a complete investigation, just a few examples.)

What's your reference for dates, I can't find a good second source to refence that includes the times.

I get them from Zack's. Here's a link to Agilent's earnings calendar from Zacks for example. If you cross reference with the company's investor relations page on their website, you'll see that when it comes to past earnings dates, Yahoo is wrong whereas Zacks is correct. On the other hand, when it comes to future earnings data, Yahoo is correct whereas Zacks is wrong.

So I get my past earnings data from Zacks but my future earnings dates from Yahoo, as a workaround until Yahoo correct their bug.

@nicolasbonder
Copy link

I'm in the US. cTicker.earnings_dates() isnt giving me anything, but cTicker.earnings_history() does. (It just doesnt show me the future earnings dates..)

I am not in the US, and this worked for me. I did:
ticker = yf.Ticker(ticker_selected)
earnings_dates = ticker.earnings_history
I can see a table with the information regarding the last 4 quarters

@ValueRaider ValueRaider unpinned this issue Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests