Why IMAP messages marked for offline viewing disappeared

[krysia]

I think my earlier query got lost in the thread I'd posted it in, so I'll try again in a new one.  Can anyone explain the following?

Although I understand that all e-mail messages stored on the server have gone, given this weekend's hardware failure, I don't understand why they have disappeared from Outlook Express too - although l've set up my mail account as an IMAP one, I marked the inbox for offline viewing, which should mean the messages are stored on my hard drive as well as on the server.  I'm really worried about this, as my old F2S IMAP account is still working but will presumably close before too long, and in anticipation I'd marked all messages in it for off-line viewing, so that I wouldn't lose them when I lose the account.  But after this experience, I wonder if I will.

[Rik]

Don't IMAP folders synchronise when you connect to the server, Krysia? In which case, the fact that they were 'deleted' from the server would mean they local copies would get deleted too. I don't generally use IMAP, but I thought that's how it worked.

Sorry if I missed your earlier post, I don't recall seeing one from you, but it has been a crazy weekend.

[Ann]

Working offline is meant for dial up only, so that when you are not connected you can work, and when you reconnect, as Rik says, it synchronises with the server automtically.  I know that from Thunderbird but I am fairly sure that it works the same with other emailers.

[Rik]

Thanks for that, Ann.

[Adam]

Marking IMAP folders for offline viewing is different than simply working offline. It is there so you can access your mail without having to connect to the IMAP server. It also enables you to keep a local backup of your mail encase the server is unavailable or data on the server is deleted.

As for the original post; there should be an option to enable viewing deleted mail in your email client. When enabling this option the mail should show up. If not, you can find the mail files on your hard drive in the Outlook application data folder.

[krysia]

Thanks for that, Adam.  I think I fouled up:  I didn't create a 'deleted items' folder when I set up the IDNet account on OE, as  I didn't think there was a use for it (given that you first 'delete' and then 'purge' unwanted items, and they don't disapppear until you purge).  I couldn't find an application data folder in OE where the messages might be stored - does it exist in OE and, if so, where should I look?

If only synchronisation had restored the messages to webmail, instead of removing them from OE!  I wonder what the chances are of IDNet being able to recover any of the lost data?

[Rik]

If only synchronisation had restored the messages to webmail, instead of removing them from OE!  I wonder what the chances are of IDNet being able to recover any of the lost data?

I know they will be trying to resurrect the RAID again tomorrow, Krysia.

[Rik]

I couldn't find an application data folder in OE where the messages might be stored - does it exist in OE and, if so, where should I look?

Go to Tools > Options > Maintenance > Store Folder, that will give you the folder, there you will find a bunch of dbx files.

[krysia]

Many thanks for that, Rik, and for keeping us all posted on what was happening over the weekend - I know how much we all appreciate it.

I've gone into OE as you suggest and found the name of the store folder, but can't figure out how to access it.  Clicking on 'store folder' just brings up a box with its name, which asks me if I want to change its location.

I don't suppose there's a way of making the same mail account both IMAP and POP3, is there?  I need to access my mail (including old mail) from various locations, which is why I like IMAP, but after this weekend, I can see the need for backup!

[Rik]

Hi Krysia

Many thanks for that, Rik, and for keeping us all posted on what was happening over the weekend - I know how much we all appreciate it.

That's kind of you, and appreciated. Unfortunately, due to the circumstances of the problem and the fact it was a weekend, I couldn't get as much information as I would have liked.

I've gone into OE as you suggest and found the name of the store folder, but can't figure out how to access it.  Clicking on 'store folder' just brings up a box with its name, which asks me if I want to change its location.

Once you have the folder name, you can navigate to it in Explorer, though you may have to change your settings to show hidden files and folders.

I don't suppose there's a way of making the same mail account both IMAP and POP3, is there?  I need to access my mail (including old mail) from various locations, which is why I like IMAP, but after this weekend, I can see the need for backup!

I've successfully used OE6 as an IMAP client, then Outlook as a POP client on the same mailbox. The critical thing is to access IMAP first. With the old idnetfreemail addresses, it was possible to forward mail, but that doesn't seem to be an option on idnet.com unless I've missed something.

The failure this weekend is almost a freak occurrence. The chances of two disks in an array failing simultaneously is tiny. Unfortunately, if it does happen, then data loss is inevitable (unless large amounts of money are thrown at the issue) as mailbox contents is changing all the time.

[krysia]

Many thanks again, Rik.  I did try going into Explorer earlier but couldn't see the folder - I didn't realise it might be hidden!  OE is obviously a lot more complicated than I thought it was.

I use Outlook at home for my work e-mail, which I access via a VPN, so I think it might be too complicated to add my home stuff to that (especially as at one point I deleted the local folders, as I never used them).  As you say, the failure of two disks is a freak occurrence, so with any luck it'll never happen to me again . . . .

[Rik]

Just a thought, Krysia, but if you were to create a new user on your home machine, you could separate the Outlook work and personal email... Alternatively, you could use something like Thunderbird to give you a backup of the boxes locally.

[krysia]

That's a really good suggestion about using Thunderbird as a local back-up, Rik - thanks!  I'd give you a karma or two for that and all your other help if I knew how! 

So, just to make sure I understand, as long as I access my account first via IMAP in OE, accessing it in Thunderbird as a POP3 account will download the messages locally without removing them from the server?

[Inactive]

That's a really good suggestion about using Thunderbird as a local back-up, Rik - thanks!  I'd give you a karma or two for that and all your other help if I knew how! 

Just click on the " good " next to karma on the forum, to add karma.

[Rik]

That's a really good suggestion about using Thunderbird as a local back-up, Rik - thanks!  I'd give you a karma or two for that and all your other help if I knew how! 

Gratefully received, thanks. 

So, just to make sure I understand, as long as I access my account first via IMAP in OE, accessing it in Thunderbird as a POP3 account will download the messages locally without removing them from the server?

That will do it, but you need to set TBird to 'leave mail on server'.

[krysia]

Sorry to bother you with more questions, Rik!  Should I set it up as a POP account right from the start, and if so, where do I find the 'leave mail on server' option?

[Rik]

Yes, set it up as a POP client from the outset. As to leave on the server, I was hoping you wouldn't ask that question, Krysia, as I never got on with TBird so don't have it on the machine any more. I can tell you for Outlook and Eudora, but that doesn't help much. 

Is there a TBird user in the house??

[krysia]

Just in case nobody answers that, I'm willing to try Eudora for my back-up POP3 account, so how would I arrange to leave mail on the server but also download it locally using Eudora?

[Ray]

Krysia, I don't use Thunderbird but I do have it on my PC

You need to go into Tools - Account Settings

Then select server settings for the account on the left.

Then tick box for leave messages on server you have a choice of leaving them for 7 days or until you delete them.

If you have more than one account set up you will have to follow this procedure for each one. 

[Rik]

It's a while since I've used Eudora, but in the account setup, there was a checkbox for 'leave mail on server'. I've just been looking at the TBird introduction and FAQs, but can see nothing about such a setting.

[Rik]

Then tick box for leave messages on server you have a choice of leaving them for 7 days or until you delete them.

Thanks, Ray - a karma is yours.

[Ray]

Thanks, Ray - a karma is yours.

Thank you, Rik. 

[krysia]

Thank you both very much indeed!

[krysia]

That's all seems to be working okay - thanks again for your patient instructions.  One more question, however:  Thunderbird only seems to have folders for inbox, unsent, and deleted items.  How do you create folders for sent items and drafts?

[Rik]

You don't.

On a POP client, the sent folder requires the item to have been sent from the client. The deleted folder is a bit like the re-cycle bin, and will generally be emptied on exit. There isn't the same relationship with the server as there is in IMAP, ie folders are purely local to the client.

The only way I can think of to do a 'perfect copy' of an IMAP setup is to make incremental backups of the OE data folder at regular intervals.

[Simon]

I have this on my laptop, but have to confess, I have never used it, however, it might be some use to you, Krysia. 

http://www.goodsync.com/

[Nick AJ]

That's all seems to be working okay - thanks again for your patient instructions.  One more question, however:  Thunderbird only seems to have folders for inbox, unsent, and deleted items.  How do you create folders for sent items and drafts?

You need to set Thunderbird to "subscribe" to the sent items folder. (right click on the account name and chose subscribed folders). Also under [tools][account settings][copies * folders] you need to set TB to place a copy of the sent message in the sent folder on the IMAP account.  This then sends a copy to the email server so it can be displayed on any device logging on to the IMAP account.

As you most likely know IMAP acts like a master copy of your e-mails, so that you can access and view messages from any PC regardless of which PC sent them.

I must admit it is very worrying that IDnet have "lost" peoples IMAP accounts, hopefully they can be recovered from ofline backups.

[Rik]

The problem is, Nick, that we're trying to get TBird to act as a POP client to provide local backup of mail for Krysia. I don't use IMAP, but some of the data loss seems to have come from synchronising the local folders with empty server folders. The plan was to get round this by having TBird grab everything, but leave it on the server. If you can think of something better, please chime in as we're engaged in some very Heath Robinson fudging here.

The data loss stems from two disks in a RAID having failed simultaneously. As you'll appreciate, conventional backups of mailboxes are never going to be able to recover data completely, given the speed at which the data is changing. I know Simon is working with the array today to try and recover any data he can.

[Ray]

That's all seems to be working okay - thanks again for your patient instructions.  One more question, however:  Thunderbird only seems to have folders for inbox, unsent, and deleted items.  How do you create folders for sent items and drafts?

Krysia, you need to go into Account settings - copies and folders and make sure each account has it's check boxes as per this image you should then have sent and draft folders.





[attachment deleted by admin]

[Rik]

I don't think they would be able to replicate the IMAP folders, though Ray.

[krysia]

Well, I spoke too soon when I thought everything was okay.  Thunderbird had started downloading messages before I could tick the box re keeping things on the server, but I quickly did so and was relieved to find that the messages were there both in my old F2S IMAP account and in the Thunderbird POP3 backup.  However, now that Tbird has finally finished downloading the messages, I've discovered my F2S inbox is suddenly empty!  Is there any way I can export the messages from Tbird back to OE? 

[Nick AJ]

Sorry guys, got things the wrong way round.

I was looking at TB being the IMAP client and Outlook being POP3. In many ways this can be better as Outlook isn't brilliant as an IMAP client.

If you are using outlook as an IMAP client you need to create a rule that copies your sent messages from the local sent items folder to the sent items folder on the IMAP account.  Outlook won't do this automatically.  Once you've done this it should keep a copy locally and also on the sent items folder on the IMAP server.

Hope this helps.

As regards the backups, from what I can gather we are talking about a two disk array on one machine?  Hardly over specified!

[Ray]

I don't think they would be able to replicate the IMAP folders, though Ray.

True Rik, I've got TB set up with a pop3 accounts and I only get the following folders under each account: -

Inbox, Drafts,Templates, Sent and Deleted

[Rik]

Sorry, Krysia. If I'd have thought, I should have warned you to drop the connection during setup, to stop TBird rushing off (as do all email clients, it seems, with the exception of Outlook).

I can't see an import facility on OE for TBird, but it might just work with the Netscape mail setting. Alternatively, what export settings does TBird have.

Keeping mail synchronised across multiple servers & machines is a nightmare. I use SynchPST for Outlook, in preference to IMAP, as it allows me to control the data. That said, to put things in perspective, I've probably received in excess of 5,000 emails from IDNet, and until Saturday, nothing had been lost. My 1&1 account was in turmoil for days in February, and I can't even estimate what was lost.

[Rik]

If you are using outlook as an IMAP client you need to create a rule that copies your sent messages from the local sent items folder to the sent items folder on the IMAP account.  Outlook won't do this automatically.  Once you've done this it should keep a copy locally and also on the sent items folder on the IMAP server.

Thanks, Nick.

As regards the backups, from what I can gather we are talking about a two disk array on one machine?  Hardly over specified!

As I understand it, it was two disks from five that failed.

[krysia]

I've been using OE as an IMAP account without any problems for a number of years - what I could use advice about now is how to get the messages downloaded into Thunderbird back into my OE IMAP account.  Presumably that's now impossible, given that the messages have been deleted on the server:  even if I got the messages into OE, synchronising the account with the server will make them disappear.  So, is there any way I can get them into my local OE folder - then I can delete Tbird and just hope for no more RAID failures!

[Nick AJ]

Well, I spoke too soon when I thought everything was okay.  Thunderbird had started downloading messages before I could tick the box re keeping things on the server, but I quickly did so and was relieved to find that the messages were there both in my old F2S IMAP account and in the Thunderbird POP3 backup.  However, now that Tbird has finally finished downloading the messages, I've discovered my F2S inbox is suddenly empty!  Is there any way I can export the messages from Tbird back to OE? 

You need to stop and take stock of where you are and what data you've got.

You can't transfer data from TB to outlook (as far as I know, but I'll wait to be corrected).

First of all take a backup of all you email data, both TB and outlook.

You could then create a new IMAP account on IDnet or F2S and use thunderbird to populate it by copying all the data from your POP3 account inbox to the new accounts IMAP inbox - hopefully it should then synchronise the two.  If you use IDnet the uploads won't count against your allowances.

Theres nothing stopping you having a POP3 and IMAP login for the same account with the same e-mail client.  Use IMAP during the day, then POP3 at night to "archive" the IMAP corrected  inbox.

[Rik]

Theres nothing stopping you having a POP3 and IMAP login for the same account with the same e-mail client.  Use IMAP during the day, then POP3 at night to "archive" the IMAP corrected  inbox.

Nice thinking, Nick.

[Nick AJ]

You can't transfer data from TB to outlook (as far as I know, but I'll wait to be corrected).

Well a google search has given a few possibilities:

http://www.broobles.com/imapsize/th2outlook.php

http://www.generetic.org/information-vault/2615-howto-export-emails-mozilla-thunderbird-outlook-2003-a.html

Plus others!

[krysia]

The problem is, I need access to old messages from diferent locations, so archiving in that way doesn't work for me.

You could then create a new IMAP account on IDnet or F2S and use thunderbird to populate it by copying all the data from your POP3 account inbox to the new accounts IMAP inbox -

I may try to create a new IMAP account on F2S - if so, how do I "use thunderbird to populate it by copying all the data from your POP3 account inbox to the new accounts IMAP inbox." 

[Nick AJ]

Ok, just had a play using thunderbird.

Create an IMAP account for your F2S email account - as it's now empty we can't do any harm.

Then choose one email in your TB inbox and COPY it to the inbox in the IMAP account.  It should then synchronise with the server and the email should be visible in the inbox using webmail.

You can do with it what you wish!

Good luck

[Rik]

Thanks for all your input, Nick. Have another karma.

[krysia]

Thanks, Nick, but I have about 1000 messages, so the one-by-one scenario isn't going to work.  Any faster way?

[Nick AJ]

Thanks, Nick, but I have about 1000 messages, so the one-by-one scenario isn't going to work.  Any faster way?

Well I only tried one - nothing to stop you doing them 100 at a time I'd guess...................

[krysia]

I need some more help, Nick.  When I click on a message and then find 'copy', it only seems to give me an option to copy it to Tbird.  Can you please spell out how I copy a whole message from Tbird and then move it to OE?  Thanks.

[Rik]

Hi Krysia, Nick's offline, so I'm going to put words into his mouth and let him correct me when he's next in.

I think the idea is to copy the mail within TBird, from local POP to local IMAP, then do a sync so that server IMAP matches local. Then use OE IMAP to get the files back to OE.

Does that make any sense?

[Nick AJ]

Hi Krysia, Nick's offline, so I'm going to put words into his mouth and let him correct me when he's next in.

I think the idea is to copy the mail within TBird, from local POP to local IMAP, then do a sync so that server IMAP matches local. Then use OE IMAP to get the files back to OE.

Does that make any sense?

100% Rik.

Or use one of the links to convert the data.

[Rik]

100% Rik.

Wow, I must be slipping. 

[krysia]

Thanks, Rik - that certainly does make sense!

[krysia]

And - whew! - it worked!  Now if someone could just suggest how I move 100 messages at a time . . . .

[Rik]

Can you not drag'n'drop?

[Nick AJ]

And - whew! - it worked!  Now if someone could just suggest how I move 100 messages at a time . . . .

Just [select all], and [copy to]

[krysia]

Happy to say all messages are now back in my F2S inbox, and I also managed to restore the ones Tevor lost to my idnet inbox, as I'd marked them for offline viewing at work too (and remembered to disconnect my connection before  starting OE!).  Thank you both again for all your help.

[Rik]

Glad you're sorted, Krysia. Computers!! 

I use an app called Migo Sync, which allows me to take my mail and desktop to any machine. A 2- 4GB flash drive is all you need to transport your own 'private' machine, it leaves no trace on the host machine when you disconnect and you can, optionally, encrypt the data.

[Inactive]

Sounds interesting Rik...

I assume it is " Free "..

[Rik]

No, it's the one I posted an offer on in Tweaks, tips etc. It was £9.99 when I bought it.

[Simon]

There's also a thing called GoodSync, from the makers of Roboform, which might be of use - and it's free! 

[Rik]

Migo Sync is more about transporting your desktop from machine to machine, Simon. I run it in the evening on my desktop machine, then if I come back online, I'll usually use the laptop, so I can just plug the flash drive in and my desktop's desktop, appears on the lappy (and I leave no trace on the lappy of my activities).

[Simon]

Yes, you mentioned it before, Rik.  I just wondered if GoodSync might be useful for synchronising mail folders, etc.  Haven't used it myself, so I can't vouch for it.  Just happened to see it on the Roboform site.

[Rik]

Me neither, Simon.