VGCollect Forum
VGCollect Site Stuff => News => Topic started by: matt on November 21, 2011, 10:17:37 am
-
What Happened?
Our long time hosting company decided late last month that they were closing down shop. Notices were sent to all customers including me, explaining the situation and giving a time table for gathering data and working through trasnfers. This was great as I was looking to update our hardware anyways and was already planning a move.
My false understanding was that we had until the end of the month to work through our migration. Well that leads us up to yesterday, Nov. 21st. Apparently this is the day the switches were flipped off, 30 days from the day the notice was sent out.
But Matt, don't you backup?
The server is setup to run nightly database and disc backup on the server as well as monthly off-site backups.
Excellent...Right?
Well here is where I break the bad news. We have lost all data from Nov. 2nd through Nov. 21st. I created a manual backup on Thursday of all files but did not include the database in this process. I attempted to grab a dump from the server but I no longer have any access to it. The offsite backup is from last month and the only manual backup I have was from Nov. 2nd, which is where we are right now.
The Future
We are now safe and sound at our new host on a new managed virtual server. Our backup process will be updated later today to manual do an offsite database dump nightly.
Summary
I screwed up and lost some user data. I feel horrible about this and will make sure to fix the mistakes that led to this. The site has grown from a small little project that I started earlier this year, to a small, awesome community and something you all rely on. I know some of you lost some data and work but right now all I can offer is my apology and a guarentee that the issues have been addressed.
If there are any other questions or comments, please drop them here or in a pm. Thanks.
Matt
-
Also you may notice a good amount of box art still missing. I'll be pushing these out over the afternoon.
-
:'( I'm glad I'm not @foxhack right now.
On the bright side, this is a good way for us to jump on our individual submission/edit numbers by pushing a whole bunch of stuff out after this little setback. The holiday weekend - a perfect time to avoid the family by obsessively cataloging our toys.
-
That sounds like a fantastic idea!
As for those who have lots of recent submissions, @scott and I may have a plan here. We have all of the box art for the items submitted. I'm thinking we can spend some time and restore those items this week. I'll post an update on this once I figure out the plan for sure.
-
Do you happen to have a cached copy of the duplicate listing thread? I see that you have been able to restore a few threads. I'd just like to see what was reported and fixed so we can fix them again.
-
That's a good question. I'll see if I find it on my hunt. (Honestly, I'm just going through google's cache feature.)
-
This is depressing. It'll be a lot of manual labor, and may take a while, but, I'm thinking I should be able to restore the games that were added between Nov 3rd and Yesterday. Until then, I might suggest holding off on resubmitting new games to the database, as when I start updating I'd be overwriting any newly submitted games with "old" games.
-
:'( I'm glad I'm not @foxhack right now.
*ahem*
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGH :(
Well, I still have -all- of the artwork I've uploaded since then. I always keep a local copy. I'm just going to hate having to do everything again... T_T I had 650-something items on my collection, boohoo.
-
.... We have lost all data from Nov. 2nd through Nov. 21st....
Yup. I signed up on November 7th. Now to readd my games and reupload some box art.
-
So it's safe to start submitting stuff again, right?
Edit: Speaking of, editing entries gives me a 500 Internal Server Error. Guess not.
-
I lost 100+ edits. not too bad. ill resubmit my front-page picture when I get home.
This is what happened to my other website but the offsite backup was over a year old and the service provider didn't give us a notice.
I hate to say I told you so...but I mentioned regular offsite backups last time we had server problems :p
also does this mean the buy and sell features need to be remade?
-
I'll post an update on this once I figure out the plan for sure.
I think the plan is that Matt and I are going to look at the box art images that we have. Each image is numbered with the same number as the item it belonged to, which will tell use exactly where in the database they went and which game it was. Then we are going to manually insert them back into place in the database and rebuild it to the point where it was before we lost the data. I was planning on starting this tonight after I get out of work at 5p EST. If all goes well, we just need to run a batch upload that will automate matching the data with it's proper item number. If it's not so simple, we input each item by hand.
Until then, I suggest waiting on submitting new items.
-
@madmax I have the frontpage picture saved on my Laptop at home, so I didn't lose that. Almost all of the images for the site are backed up twice.
-
I think the plan is that Matt and I are going to look at the box art images that we have. Each image is numbered with the same number as the item it belonged to, which will tell use exactly where in the database they went and which game it was. Then we are going to manually insert them back into place in the database and rebuild it to the point where it was before we lost the data. I was planning on starting this tonight after I get out of work at 5p EST. If all goes well, we just need to run a batch upload that will automate matching the data with it's proper item number. If it's not so simple, we input each item by hand.
Until then, I suggest waiting on submitting new items.
Then you should probably pull the plug on the "submit new item" button for the moment. Otherwise you'll get people like me messing with stuff without meaning to.
-
@foxhack Nevermind, Matt just told me to "Let them go for it" ha. So Start resubmitting items to the database asap. ha
-
on the bright side it was only about 3 weeks worth of data. it could have been a lot worse. Hopefully this doesn't discourage those who worked hard during those 3 weeks :/
-
on the bright side it was only about 3 weeks worth of data. it could have been a lot worse. Hopefully this doesn't discourage those who worked hard during those 3 weeks :/
Good thing I stopped working hard about 3 weeks ago. ;P
-
In my case it was around 85 new / edited Steam entries. So yeah, I'm ticked. :P
-
Some images seems to be broken now?
http://vgcollect.com/item/5305 is an example.
ifg you do a search for "Final Fantasy" In the search bar, a lot more images are coming up like this.
-
Oh hey, we're gonna get to hit the 2,000 users and 19,000 items mark.... Bonus?
:/
-
@madmax We are still missing about 5000 images. I'll be pushing out those as soon as I can.
-
We were way past 20,000 listings before the loss.
-
Actually we weren't. Those item numbers are off by about 1,000. Thanks to dupes and merges.
-
Ah.
Well I just got a server error when adding a game so I'm gonna hold off for now. :p
-
@madmax We are still missing about 5000 images. I'll be pushing out those as soon as I can.
Guess we should hold off edits until these have gone through :)
-
Yep, for example The Binding of Issac is item number 19966. But we only have 18854 listings, which means we are currently off by 1112 items. Its a database issue, the more dupes folks submit the further off we go unfortunately.
We had talked about replacing missing numbers with newly submitted items. But matt did some research and from what he found, database experts say its not worth the effort and just let the empty numbers hang.
-
Man am I glad I save all my box scans... Back to work resubmitting the group of PC games I got recently ;)
-
But matt did some research and from what he found, database experts say its not worth the effort and just let the empty numbers hang.
That, definitely. It's a huge hassle.
Good to know the situation. It's unfortunate, but it's definitely not as bad as it could have been... I would know. I just find it weird that you didn't get a "last notice" of sorts a day or 2 before, it's definitely strange for them to just leave you hanging like this. Well, I hope the new server is gonna be good.
-
All my Atari 5200 entries have been deleted from my collection, and trying to go thru all other collections to see if anything is missing from them. Hopefully we won't have to go through this again. Oh well if we all pull together we should get this all back up and running in no time. I will start on Atari 5200 images sometime this week, good thing I have back up of all of them.
-
Whoever's adding new entries, the images refuse to load, so my suggestion to stop people from adding new stuff stands. Especially since whoever's adding the Steam stuff is doing it wrong -_-;;
-
Strange, cause I did 2 image edits earlier and both of those images are showing up... hmm.
-
Just did some testing and Matt says everything should be working. So feel free to edit and submit. He's in the process of fixing all the broken images right now.
-
It's the No Art file that's missing. :(
And I still can't add images to some entries. Like http://vgcollect.com/item/19967
-
Hmm. I edited it just fine. w t f.
-
Looks like your edit went through too @foxhack. I approved yours and deleted mine.
-
... So it seems that any correction I submit gives me a 500 Server error. :( I tried to fix http://vgcollect.com/item/6253 and http://vgcollect.com/item/16497 so the site hates me.
-
You aren't crazy. I just got another one, first one in a bit too. Matt's looking into it now/again. Get that pitchfork ready Fox...
-
Hmm this one is driving me nuts. Error logs aren't giving up any good clues either.
Good news though...all of the missing images are now being uploaded. Only 35,000 to go!
-
Only 35,000 to go!
Ha yay!
On an edits note, I've done three extensive edits in a row and each one has gone through just fine...
-
Ditto, three edits and two new submissions. All fine.
-
UPDATES
- Almost all backup box art is now loaded. About 7000 more files as I type this.
- I've setup a new offsite nightly database backup. The data is actually dumped out to my Dropbox account, so I'll have a nightly snapshot which will cover us in the future.
-
Yeah, I'm noticing alot of stuff missing so I'm helping to update box art and some of the data.
-
Just got another 500 server error. I submitted http://vgcollect.com/item/20053 and the info went through but the image did not, so these errors might be related to the images.
-
It seems like it is an upload issues but it's so random that I haven't been able to track things down. Thanks for reporting it though. It'll help me narrow things down.
-
I just submitted Frogger 3D and the images didn't take. I scanned the box and produced two 2.5 MB jpgs.
-
@andrew could you send me those images to matt@vgcollect.com. I'll use them for testing. Thanks!
-
@matt will do. None of my scans (all around this size) went through. I also submitted a couple for edits, but I suspect those didn't go through either because my corrections went though, but not those.
-
I finished going through my collection (except for PC and C64) in early November :( Lost 38 entries in my collection, not sure how many were adds and I'm afraid I'm probably not going to go back to it for quite a while. Only reason I was doing a lot in late October and early November is because my parents were visiting so I had a laptop available. That made it a lot easier since my games are upstairs and my PC is downstairs. I didn't submit any images though, just game entries. Probably edited some too but I can't remember which ones. It was likely stuff for PS2, Xbox and anything newer.
-
There's a way to get the info... it's kinda glitchy on Firefox, but check your internet history! Sort it by location and date, and you'll be able to see, at least, a list of all items you had browsed through or added.
-
Just wanted to congrat everybody. We are back over 19,000 games again! Thanks for all your hard work and dedication. :)
-
I do not know if it is related to the server outage, but when i click on the edit icon next to a game in my collection it appears in a box so small that you can only see the edge of the box where you can type stuff.
-
@andrew could you send me those images to matt@vgcollect.com. I'll use them for testing. Thanks!
@matt is there any update to this? I have several large scans that are ready to be submitted.
-
@andrew No update yet. I'll run some tests tonight and report back. I'm about 75% sure it's just a file size issue and I need to up the max limit the server will handle.
-
@andrew Please give it a try now. The max uploads were set to 2 megs. I've bumped this up quite a bit and we should be able to handle anything under 10 megs now.
-
@matt Excellent! I'll begin submitting... right now.
-
A collection of things - firstly, that's a shame, but nothing too horrible, I'm sure you can recover.
Secondly, the phantom person who posted a thread about an SQL error was me, and yes, it was probably due to old cookies.
-
Welcome back @pie!
-
Welcome back! As you noticed your account was one of the casualties. :( Fortunately new methods are in place to prevent that from ever happening again. Hopefully it won't take too much work to get yer collection back up to speed. Let us know if you need anything.
I'm also glad to know that you were the phantom! I looking into it and couldn't find anything other than a user account in the forum with a blank username. So confused...