Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EML-Import: Missing images, missing attachments #655

Open
Pierre-de-la-Verre opened this issue Nov 17, 2024 · 34 comments
Open

EML-Import: Missing images, missing attachments #655

Pierre-de-la-Verre opened this issue Nov 17, 2024 · 34 comments
Assignees

Comments

@Pierre-de-la-Verre
Copy link

I exported hundreds of EML from Outlook 2010, using CoolUtils. Many of them have attachements (PDF, Images, ..) and embedded images.

  • When I open an EML with Thunderbird, everything is OK. I see and get what I want.
  • When I "import" the files to a local folder, I see the mail body, the placeholder for embedded images, the name of attached files, the correct size of the entire mail - but all images and attachments are empty. I don't see them, I can't save them, I can't open them.

I have Win 10, 64 bit, German, current Thunderbird and current Add-In (14.1.2). Do I have a wrong setting? What can I do?

regards!

@cleidigh
Copy link
Collaborator

@Pierre-de-la-Verre
I have a beta for the next release which has some fixes for eml imports. While the fixes are not explicitly for attachments, it would be good to try first.
You can grab here:

@cleidigh

@cleidigh cleidigh self-assigned this Nov 17, 2024
@JensFreund
Copy link

I have exactly the same issue. I uploaded a huge amount of old mails in eml format. Import was fast and complete. But all the imported attachments can neither by displayed nor be saved (at least JPG and PDF are affected). When I click on mails with JPG attachments for the fraction of a second the picture is visible in the details window and afterwards it is disappearing. Attachments from newly received emails are readable (it only affects the imported ones). I imported eml from an old windows mail client. I have new windows 11, also German. Also current thunderbird. (just installed for the first time on a new laptop).
Evyrthing else is working. (However the file with my 2000 uploaded sent items is 18 GB big, which is at least a factor of 10 too much for the content, but I do not know if the errors are related).

@cleidigh
Copy link
Collaborator

cleidigh commented Nov 22, 2024

@JensFreund
Can you try my beta b6 for v14.1.3.
It has couple eml fixes, though I'm not sure they will address this.

If you can id a single eml message that has this issue that you could send me, that would be helpful.
Use:
test1@kokkini.net

@cleidigh

@JensFreund
Copy link

I have sent a folder with some emails with attachments to the email you mentioned.
The issue does not appear, if I open a single eml with thunderbird. The issue also does not appear if I upload a single eml with your add-on into a local folder. The issue appears, if I upload a complete folder (or a folder with subfolders) with the addon into a local folder.

@JensFreund
Copy link

With the version in the link above it works (at least with one folder)! I can see jpg and pdf. I will try a full upload.

(have to find a workaround for another issue upfront)

It took me quite a time, since I did not find any download button. :-) -- For beginners to thunderbird:

  1. follow the version link
  2. click on the link "latest version"
  3. click on "raw" (whatever this means :-)) --> you get an XPI-file in your download folder
  4. In thunderbird Menu --> Extras --> add-ons and themes (this will bring you to the Add-On Administration tab)
  5. Click on the settings icon on the top (The icon is called "tools for all addons") --> Install add-on from file
  6. browse to the XPI file and install it --> the version number will get updated (You can see this in the add-on administration by clicking on the add-on)

@cleidigh
Copy link
Collaborator

@JensFreund
I didn't receive your email.
Is it possible it bounced?

@JensFreund
Copy link

I had a typo in your email address and I did not receive any nont-deliverd mail. I will send it again.

@JensFreund
Copy link

Sorry -- I am still not used to the new keyboard of my new laptop :-)

@cleidigh
Copy link
Collaborator

@JensFreund
Got your zip!
If my beta worked for one folder I am hopeful it will also be good for subfolders.
Did you try on your full set?
@cleidigh

@JensFreund
Copy link

I first have to fight to get a handfull of new mails out of an inflated folder created from the last upload. (The file size of the folder is now 18 GB which causes thunderbird to not react on some commands like moving to another folder). The only thing I can currently do is deleting the complete folder, but I do not want ot delete the new emails. --- This can take a while....
Then I will delete all my last uploads (more than 15 years of private mail history), remove it from the recycle bin and then upload it again from the same input as last time.
But I can make a test upload, of a smaller folder set.....

@cleidigh
Copy link
Collaborator

@JensFreund
Ok, I am also hoping the inflated sizes were also a byproduct of the main issue.

@JensFreund
Copy link

2000 mails in a folder with 11 subfolders worked. I can read all the pdf attachments and one txt attachment.

@JensFreund
Copy link

The inflated folder was my biggest folder and it was the only sent items folder (if this is a hint)

@cleidigh
Copy link
Collaborator

@JensFreund
Ok great so your full import now works as expected with the beta, that's good.
Do you need to try the Sent folder to verify it doesn't create an inflated size?

@JensFreund
Copy link

this was not the sent items folder yet. That was only a small subset of my mails. I will try the sent items folder next

@JensFreund
Copy link

It is much slower than the other upload. I got stuck at 1842 of 2006 emails. It simply stopped. But the file size is now a bit more than 10 times smaller 1.29 GB for 1842 mails (was growing porportionally with the mails). Attachments are readable.
I can move mails out of this folder. So I will split it (once I manage to get a full upload) to keep file size below 500 MB.

Probably it will all working if I delete the inflated folder.
I have too look first how many new mails I have in (probably only 1-5) before deleting it.

@cleidigh
Copy link
Collaborator

@JensFreund
Once you clean and retry let me know if there are any problems.

@JensFreund
Copy link

The 1842 mails are ok. It did not stop. (Looks I merged mails from another send items folder in it to get the 2006). So it all works fine. (sorry for the confusion, but I am not working on a planned test case, but on my real data :-)). It still was slower, but this is probably only because there are more attachments
I only have 1 mail and this mail is not important. So I can delete the inflated folder (and later all the others, but there I have a bit more recent mails, which I need to move first). ... before I can do the full upload again

@cleidigh
Copy link
Collaborator

@JensFreund
Ok.
I have had reports of slow eml imports. Unfortunately this appears to be a Thunderbird issue with adding messages.

@JensFreund
Copy link

I just see that I can distinguish new and old mails not only by the date, but by the fact that I can move the new mails into another folder, while I cannot move the old mails in the old folder (uploaded with the version with the bug). ...... Since I only want to move the new mails, it is ok :-)

@cleidigh
Copy link
Collaborator

Good!

@JensFreund
Copy link

Only 2 new mails overall got stuck in the old folders. I cannot save them (like I cannot save the old mails). I also cannot move them. But I can forward them to myself. This operation makes the attachments readable again (but then you have the wrong sender of course).
So: forwarding a mail with a corrupted attachment makes the attachment readable again. (In case this is helpful for somebody, in case you do not have the original eml files anymore).

@JensFreund
Copy link

Upload of all my about 14000 emails in option folder (in two tranches one with nearly 99% of the mails) with subfolders worked. Filesize is as expected. Attachments are visible. Now I can move individual mails from one folder to another. (Moving folders also worked in the past).
So with the new version everyhting seem to be ok.
Upload statistic in the progess status was 2 errors. (No idea how I can see those -- and I do not really care.)
(I just lost some hours of restructuring my mails which I did before I realized that the attachments are corrupted.).

@cleidigh
Copy link
Collaborator

@JensFreund
Excellent, 14k!
So it sounds like the beta addressed all issues.
I have to deal with one separate issue and then I really need to get this released.
Thanks!
Cheers
Christopher

@Pierre-de-la-Verre
Copy link
Author

Hi Christopher and Jens
great that you invested a lot of energy into this topic. For me, I wanted to test it later, but it seems that you already fixed it.

@JensFreund
Copy link

So far everything is working fine besides one smaller issue:
I had to copy about 100 recent mails from different folders into new folders. The old uploaded mails (with the broken version of the add-on) in the old uploaded folders (with the broken version of the add-on) could not be moved. Then I deleted all uploaded folders. Only the manually created folders remained as local folders.
After upload with the new version of the add-on I created completly new folders and moved all uploaded emails and all the mails from the new manually created folders. Unfortunately there was in 2/3 of the folders always just one new mail that could not be moved into any other folder. Those blocked mails had no attachments. I could not see any pattern, besides that it was always just one mail per folder. (often the oldest mail but not always.)
The only operations I could do with those mails is forwarding them to myself. I could not delete the mail. But I can delete it by deleting the complete folder.
--> maybe it is a side effect of the corrupted data and will not happen to any other user that never used the corrupted version.
If I see again random mails that are blocked in a folder (no move with drag and drop, no move with right-click menu, no delete, and no export to eml), I will inform you.
I hope it is closed for me now :-)

@cleidigh
Copy link
Collaborator

@JensFreund
@Pierre-de-la-Verre
Yes, I think the emls imported with 14.1.2 have database issues that prevent them from being handled correctly beyond just the attachment issue.
@cleidigh

@JensFreund
Copy link

@cleidigh
The issue I have looks different, but is probably not important (Only interesting if it repeats with other users or with me)
clarification: I know it sounds weird, but here is the situation:

  1. I uploaded a lot of folders and mails with the broken version of the add-on
  2. I got new emails (NM) received via my email account. They where not uploaded via the add-on.
  3. I moved many of them (NM) into different uploaded local folders (that contained a lot of corrupted uploaded mails)
  4. I did the update of the add-on to the version you indicated here
  5. I moved the newly received emails (NM) into different newly created local folders (NF) (did work for nearly all of them)
  6. I deleted all the uploaded local folders with all the corrupted emails within
  7. (restart of the computer)
  8. I uploaded all the folders and the roughly 14000 emails again into local folders. (with many subfolders (UF) created by the upload).
  9. Since the subfolders rather showed a historical pattern and not exactly what I see in my windows live mail account, I did not trust those folders. (That is an issue I have with windows live mail exports for some years. The folder structure in the export does not match to the structure on the UI) --> Hence I created new local folders (XF) one by one and moved all the uploaded emails into those (not moving folders). This worked.
  10. I deleted all the empty uploaded folders.
  11. As a last step I wanted to move the newly received mails (NM) from the folders NF into the folders XF. This did not work 100%. For the majority of folders NF there was exactly 1 email that could not be moved (or deleted).
    Hence some mails that were newly received via the email account and not uploaded got corrupted (either because the add-on was still the wrong version or because they were moved into corrupted folders from the upload)

@Pierre-de-la-Verre
Copy link
Author

Hi,
what's the current state of this issue? Is it fixed in a stable mode or does it need more testing? I see that 14.1.2. is still the current version - is there a schedule to release a new version?

@cleidigh
Copy link
Collaborator

@Pierre-de-la-Verre
This thread has a couple of different issues, not sure what you are experiencing.
I am working on a new release and I already have beta with some eml import fixes, not clear if it addresses all issues.
Grab here:

@cleidigh

@Pierre-de-la-Verre
Copy link
Author

I just imported 22'000 EML again and the problems I reported before are gone - the attachments in the integrated images are here and can be used and displayed.

Great work! Thanks a lot!

@cleidigh
Copy link
Collaborator

@Pierre-de-la-Verre
Thanks!
That's a nice big import.
Just to be clear, this is with the new v14.1.3 beta?
@cleidigh

@Pierre-de-la-Verre
Copy link
Author

Yes, with the version downloaded yesterday.

@cleidigh
Copy link
Collaborator

Excellent. Thanks for your testing and patience.
Hoping to release this shortly.
@cleidigh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants