Missing BTC-USD when time zone and exchange time zone are on different days

[AndrewBworth]

Hello, I've been trying to fix a problem with regards to downloading cryptocurrency exchange data. My program stops working correctly for 4 hours every day (and it's the best 4 hours, immediately once prices for a day are finalized). For example, If I want to get data for today and 2 days previous, and today is the 31st, when the time zones are on the same day, I get data from the 31st, 30th, and 29th. If the time is such that I'm on day 30th, the exchange is on 31st, I won't get data for the 30th, it will be missing. If I get the data as hourly instead of daily, there is no missing data. Here's what I get when the dates don't match
Open High Low Close Volume Dividends Stock Splits
Date
2023-08-29 00:00:00+00:00 58.262775 60.506226 56.723537 59.522854 73819166 0.0 0.0
2023-08-31 00:00:00+00:00 57.768280 57.782700 57.529903 57.529903 60968796 0.0 0.0

The hourly is OK
Open High Low Close Volume Dividends Stock Splits
Datetime
2023-08-30 00:00:00+00:00 59.523426 59.602249 59.195923 59.195923 0 0.0 0.0
2023-08-30 01:00:00+00:00 59.198109 59.198109 58.862041 59.025242 68208 0.0 0.0
2023-08-30 02:00:00+00:00 59.026306 59.096844 58.845150 58.871193 379296 0.0 0.0
2023-08-30 03:00:00+00:00 58.868065 58.868065 58.595074 58.758812 0 0.0 0.0
2023-08-30 04:00:00+00:00 58.754021 58.833527 58.754021 58.830929 362272 0.0 0.0
2023-08-30 05:00:00+00:00 58.831070 58.919514 58.831070 58.919514 359856 0.0 0.0
2023-08-30 06:00:00+00:00 58.921860 58.946293 58.660313 58.663052 265960 0.0 0.0
2023-08-30 07:00:00+00:00 58.659431 58.856384 58.635784 58.856384 727656 0.0 0.0
2023-08-30 08:00:00+00:00 58.855297 58.966709 58.837498 58.933662 420960 0.0 0.0
2023-08-30 09:00:00+00:00 58.937099 58.937099 58.505478 58.638653 706024 0.0 0.0
2023-08-30 10:00:00+00:00 58.638775 58.768738 58.626305 58.703690 0 0.0 0.0
2023-08-30 11:00:00+00:00 58.705708 58.705708 58.418526 58.418526 728832 0.0 0.0
2023-08-30 12:00:00+00:00 58.411537 58.514336 58.280537 58.396126 1865472 0.0 0.0
2023-08-30 13:00:00+00:00 58.389526 58.546192 58.243034 58.326962 2119880 0.0 0.0
2023-08-30 14:00:00+00:00 58.324394 58.403522 57.692387 57.692387 0 0.0 0.0
2023-08-30 15:00:00+00:00 57.693703 57.880993 57.521603 57.814522 0 0.0 0.0
2023-08-30 16:00:00+00:00 57.814396 57.814396 57.640583 57.693359 0 0.0 0.0
2023-08-30 17:00:00+00:00 57.690128 57.899757 57.664425 57.767086 0 0.0 0.0
2023-08-30 18:00:00+00:00 57.769123 57.769123 57.575951 57.705013 207900 0.0 0.0
2023-08-30 19:00:00+00:00 57.705021 57.749737 57.648502 57.740974 0 0.0 0.0
2023-08-30 20:00:00+00:00 57.733955 57.833313 57.661510 57.796429 0 0.0 0.0
2023-08-30 21:00:00+00:00 57.796104 57.843842 57.685474 57.699722 0 0.0 0.0
2023-08-30 22:00:00+00:00 57.698357 57.761677 57.669506 57.760857 0 0.0 0.0
2023-08-30 23:00:00+00:00 57.762268 57.793472 57.757660 57.793472 0 0.0 0.0
2023-08-31 00:00:00+00:00 57.765347 57.783474 57.513493 57.514847 2455020 0.0 0.0

Here is my code approximately, I am on Eastern time, the exchange is on UTC:

from datetime import datetime, timedelta, timezone
import pandas as pd
import time
import yfinance as yf

aaveusd_obj = yf.Ticker('AAVE-USD')

datetimenow_eastcoast = datetime.now(timezone.utc) - timedelta(days = 0, hours=0, minutes = 0)
datetimestart_eastcoast = datetime.now(timezone.utc) - timedelta(days = 2, hours=0, minutes = 0)
datetimenow = datetimenow_eastcoast #+ timedelta(hours=timezonehours, minutes = 0)
datetimestart = datetimestart_eastcoast #+ timedelta(hours=timezonehours, minutes = 0)

aaveusd_obj_data = aaveusd_obj.history(interval='1d', start=datetimestart, end=datetimenow)

print(aaveusd_obj_data)

[ValueRaider]

What exactly is the problem?

• "yfinance has bad/missing data" - impossible, yfinance doesn't have data
• "Yahoo has bad/missing data" - does it?

And tip: let yfinance manage timezones: history(interval='1d', start='2023-08-30')

[AndrewBworth]

If I let yfinance manage the time zones, the data is missing just the same. yfinance is fetching me data with missing values, is that more strictly correct? I checked the Yahoo Finance history web page, and yes, the dating is missing on the Yahoo web page. I suspect I halfway know what's wrong. Yahoo Finance is not giving me data with missing days on my cell phone, and probably not any other computer in the house. I hit a problem a few months ago, I downloaded a web page with bitcoin data from what I was told was a small cryptocurrency exchange; it was actually a ponzi scheme. They did something to my computer, it has all these problems now: CORS error, and maybe not fetching certain data when there's some kind of discrepancy on what day it is. I've looked all over, but I haven't been able to find out how to fix it. I ran a virus scan with ClamAV, which came up as no viruses

[AndrewBworth]

Here's a picture of a window I have open from last night, you'll notice August 30th is missing, and I have the page inspected, and the console tab says I have "ERR_CERT_DATE_INVALID. I don't know exactly what that means, but I suspect that's my problem. I don't know how to repair my computer. There are browser extensions that say they can fix it, but really I think that's the wrong way, although I am using that now

[ValueRaider]

The problem is almost-certainly Yahoo despite your malware adventure #925

Maybe history(repair=True) can fix it? If not, now you got a real adventure.

[AndrewBworth]

I suspect the problem is on my end, I've been having problems with my computer for a long time. The only browser that doesn't have CORS error problems on my computer is Chrome, and I had to use an extension to make it work. I tried installing 6 other browsers, and they all did not work. I don't think anyone else is having problems with missing data on yfinance, and even my cell phone can get the data. I don't know what to do though, I can't find any resources about repairing this

[AndrewBworth]

You were right, I changed my timezone on my computer to UTC, it is now 9 pm in Pennsylvania, a Yahoo Finance is missing August 31st on the browser on both my computer and my cell phone. I don't know when it corrects itself, but it's before midnight. So the problem is Yahoo, is there anything I can do, or do I have to rewrite my programs with a new API?

[AndrewBworth]

Holy cow my friend, repair = True actually did the job, unbelievable. This is pretty inspiring. I'm a amateur programmer, and I've been wanting to get involved with something for some time

[ValueRaider]

You can get involved 'cos I want a little something in return for my repair feature - improve awareness. Propose a change to README/docs that would have avoided this entire discussion.

[AndrewBworth]

I don't know, I do see that the issue has come up 3 times on this board at least, but my searches before posting didn't find them. I'll think about it