Forum Helpdesk

New hosting and software update incoming

Reply
Page 2 of 2
  Tools
Gruso +

Gruso's Avatar
Joined
Aug '04
Times thanked
<
Thanks: 9,121
Thanked 20,931 Times in 6,226 Posts
Posts
37,567
You've basically been invited to, for any stuff that's worth saving. As long as you don't start bumping squillions.

Quote:

Originally Posted by Gruso View Post

... if you would like an old thread to remain accessible after the migration, it will need some new posts in it. Just sayin'. Please be sensible.

General site status update while I'm here: NFI
Dr Rudi +

Howzit!

Dr Rudi's Avatar
Joined
Jan '14
Times thanked
<
Thanks: 1,176
Thanked 1,843 Times in 721 Posts
Posts
2,086
Is the pre-release alpha test forum for us GCP&G regulars ready yet?
Bye now!
Bracko +

Bracko's Avatar
Joined
Apr '02
Times thanked
<
Thanks: 1,532
Thanked 2,799 Times in 1,577 Posts
Posts
42,178
you mean the whole forum's regulars?
step on my cubes.
RunningWithScissors +

COMMUNIST PARTY OF ITM

RunningWithScissors's Avatar
Joined
Apr '07
Times thanked
<
Thanks: 25,255
Thanked 20,415 Times in 7,540 Posts
Posts
26,000

Quote:

Originally Posted by Dr Rudi View Post

pre-release alpha test

lol if u think itm would do that

i know the first three numbers
Bracko +

Bracko's Avatar
Joined
Apr '02
Times thanked
<
Thanks: 1,532
Thanked 2,799 Times in 1,577 Posts
Posts
42,178
We did, once.
Gruso +

Gruso's Avatar
Joined
Aug '04
Times thanked
<
Thanks: 9,121
Thanked 20,931 Times in 6,226 Posts
Posts
37,567
Pinged HQ the other day. No updates yet.
Mickstah +

"Science needs to be applied to beds"

Mickstah's Avatar
Joined
Jun '02
Times thanked
<
Thanks: 3,850
Thanked 4,159 Times in 1,819 Posts
Posts
25,889

Quote:

Originally Posted by Dr Rudi View Post

Is the pre-release alpha test forum for us GCP&G regulars ready yet?

There's probably about 16 of us. Not bad IMO.
ಥ_ಥ - - - - - - - - - - - -ಠ_

"She was like a candle in the wind... unreliable."
Dean Learner
Trase Love:
O Mick you are so cool
More cooler than a dolphin
Let's hug me


custaro +

Thirsty Critter

custaro's Avatar
Joined
Jun '01
Times thanked
<
Thanks: 898
Thanked 743 Times in 229 Posts
Posts
7,544
Removing a rich, publicly-available, Google-indexed archive of Australian dance music events, information and discussion, generated by tens of thousands of people, for the sake of the handful of users left here, is stupid.
Seamus Ryan +

Resident Perve

Seamus Ryan's Avatar
Joined
Jul '05
Times thanked
<
Thanks: 2
Thanked 14 Times in 11 Posts
Posts
4,030

Quote:

Originally Posted by Gruso View Post

[list][*]Removal of all threads that have not been posted in since January 1 2007.
.

Wait. WHAT

SHIT. *begins frantically searching for all of the classic threads*

I know I dont post here that much anymore, but I still read stuff daily, and regularly look back at old funny threads for nostalgic reasons.

Wait. wait. WAIT FOR ME

Is there really nothing we can do to save the old posts/threads? There is SOOO much content there. Cant we archive it all in a plain-text format or something so it isn't clogging up the DB?? . Export the lot of them to html and object store (ie AWS S3) the fuck out of it.
PLEASE dont delete them all.

Does this also mean our post count will drop? :-(
ARMIN

Quote:

Originally Posted by RunningWithScissors

hardstyle does have melodies they're just really retarded

Last edited by Seamus Ryan: 14-Jul-15 at 05:10pm

kris-ko +

KoThnxBai

kris-ko's Avatar
Joined
Jan '05
Times thanked
<
Thanks: 143
Thanked 321 Times in 230 Posts
Posts
3,949
Please bump/sticky for Bris - http://www.inthemix.com.au/forum/sho...d.php?t=114823 - not quite as important as the blacking out windows thread, but a close second.

:offchops:
Words I've Written
Skins (free downloads of deep/tech/prog house originals)
Fromage Of Reason (DJ mix of house reboots of pop classics - Hall & Oates, NKOTB, et al)
Gruso +

Gruso's Avatar
Joined
Aug '04
Times thanked
<
Thanks: 9,121
Thanked 20,931 Times in 6,226 Posts
Posts
37,567

Quote:

Originally Posted by Seamus Ryan View Post

Wait. WHAT

SHIT. *begins frantically searching for all of the classic threads*

I know I dont post here that much anymore, but I still read stuff daily, and regularly look back at old funny threads for nostalgic reasons.

Wait. wait. WAIT FOR ME

Is there really nothing we can do to save the old posts/threads? There is SOOO much content there. Cant we archive it all in a plain-text format or something so it isn't clogging up the DB?? . Export the lot of them to html and object store (ie AWS S3) the fuck out of it.
PLEASE dont delete them all.

Does this also mean our post count will drop? :-(

I've got no control over it man But there is a full db backup. Bump some shit though, fire up. If any good threads are locked, post links and I'll bump them.

Quote:

Originally Posted by kris-ko View Post

Please bump/sticky for Bris - http://www.inthemix.com.au/forum/sho...d.php?t=114823 - not quite as important as the blacking out windows thread, but a close second.

:offchops:

Last post in 2011 - she's safe.
RunningWithScissors +

COMMUNIST PARTY OF ITM

RunningWithScissors's Avatar
Joined
Apr '07
Times thanked
<
Thanks: 25,255
Thanked 20,415 Times in 7,540 Posts
Posts
26,000

Quote:

Originally Posted by Seamus Ryan View Post

Wait. WHAT

SHIT. *begins frantically searching for all of the classic threads*

I know I dont post here that much anymore, but I still read stuff daily, and regularly look back at old funny threads for nostalgic reasons.

Wait. wait. WAIT FOR ME

Is there really nothing we can do to save the old posts/threads? There is SOOO much content there. Cant we archive it all in a plain-text format or something so it isn't clogging up the DB?? . Export the lot of them to html and object store (ie AWS S3) the fuck out of it.
PLEASE dont delete them all.

Does this also mean our post count will drop? :-(

just scrape them yourself, gosh
i know the first three numbers
Seamus Ryan +

Resident Perve

Seamus Ryan's Avatar
Joined
Jul '05
Times thanked
<
Thanks: 2
Thanked 14 Times in 11 Posts
Posts
4,030

Quote:

Originally Posted by RunningWithScissors View Post

just scrape them yourself, gosh

Sounds like a plan. Let me get back soon.

Gruso, hypothetically if a method were produced to archive all the things in such a way that they:
A) Cost ITM nothing
B) Dont consume DB resources
C) Are made freely available to anyone (ie no ads)
D) Does not in any way infringe on ITM trademark/copyright content (ie it is an exact copy of the many threads on ITM as they are shown today in our browsers) with full credit going to ITM.
E) Is not modified in any form but simply flat-filed and presented on the internets

Do you believe the ITM peeps would be open to that? Can you pass it on?

IF this could be done, happy to dedicate my time/resources/expertise/money to make it happen. ITM is a great community, I have met so many amazing people through it and it was ultimately what gave birth to the best years of my life (to date). I would hate to see so much of it washed away knowing I perhaps could have done something to save it :-(

xox
Seamus
ARMIN

Quote:

Originally Posted by RunningWithScissors

hardstyle does have melodies they're just really retarded

Mickstah +

"Science needs to be applied to beds"

Mickstah's Avatar
Joined
Jun '02
Times thanked
<
Thanks: 3,850
Thanked 4,159 Times in 1,819 Posts
Posts
25,889
Seamus you are a god amongst men. If you are able to preserve the history of ITM (there was a LOT of great stuff pre-2007) that would be amazing!

There's a solid decade of dance music activity recorded here, it'd be a shame to see the lions share from the peak of the website disappear.
ಥ_ಥ - - - - - - - - - - - -ಠ_

"She was like a candle in the wind... unreliable."
Dean Learner
Trase Love:
O Mick you are so cool
More cooler than a dolphin
Let's hug me


RunningWithScissors +

COMMUNIST PARTY OF ITM

RunningWithScissors's Avatar
Joined
Apr '07
Times thanked
<
Thanks: 25,255
Thanked 20,415 Times in 7,540 Posts
Posts
26,000
would've been nice if they gave us similar warning before they deleted all the other parts of the site for no reason aye




e: just remembered there was that one bit that was deleted because of me though so i guess i kind of should have expected that one lol
i know the first three numbers
Mickstah +

"Science needs to be applied to beds"

Mickstah's Avatar
Joined
Jun '02
Times thanked
<
Thanks: 3,850
Thanked 4,159 Times in 1,819 Posts
Posts
25,889
Yeah but that wasn't really your fault, just a massive bug in the system that they couldn't be arsed fixing.

still amuses me that ITM once had blogs and email.
ಥ_ಥ - - - - - - - - - - - -ಠ_

"She was like a candle in the wind... unreliable."
Dean Learner
Trase Love:
O Mick you are so cool
More cooler than a dolphin
Let's hug me


Seamus Ryan +

Resident Perve

Seamus Ryan's Avatar
Joined
Jul '05
Times thanked
<
Thanks: 2
Thanked 14 Times in 11 Posts
Posts
4,030
Dudes,

I am still working on a way to do this. To keep it short, I can fork out the server resources to "crawl" every thread that is currently viewable on ITM but there are a few uncertain things I am not 100% sure of.

1. Would ITM allow it? (instant show stopper if that is the case)
2. Just how much space is required (keeping in mind all the images/uploads/gifs/avatars etc can take up a fair chunk of space). Long term the amount is irrelevant as ill probably just dump it on S3, but for the crawl itself i will need to know what will be required.
3. How to program some logic into the crawl that knows the # of pages per thread, and to crawl said pages (i am no web dev program haxor, so not 100% sure on this part) FYI, wget with several variables seems to be the way to do this.

RE question 1. I had a thought, Gruso if you could pass this on that would be great. I am guessing the bulk of the issues come from table locking whenever a user posts/updates data? If that is the case, would it be possible to drop the entire ITM archive pre 2013 or whatever into a "Archive" forum that is 100% read only. And I dont mean just create another forum within ITM. I mean drop it into its own DB + vBulletin installation, separate from the current ITM forums. Lock it up, make it read only and sit there with no post/user updates at all. Disable registrations, effectively preventing any further updates. Allow searching of course, but that is about it.
This would:
A) Allow us to look back over old threads, remember the good (or bad?) times and get a laugh every so often
B) Allow us to continue running on spanking new forum with only 2013/relevant threads in it.

Again, happy to provide resources if required.

Cheers,
Seamus
ARMIN

Quote:

Originally Posted by RunningWithScissors

hardstyle does have melodies they're just really retarded

Gruso +

Gruso's Avatar
Joined
Aug '04
Times thanked
<
Thanks: 9,121
Thanked 20,931 Times in 6,226 Posts
Posts
37,567
Awesome of you to offer this Seamus. I don't want to lose our history either.

I don't think HQ would have a problem with it. I've been told there was / will be a full db backup taken - pre-2007 and all. Getting hold of that would be much better than a scrape. (A scrape is doable, but I have NFI about the time needed to do it, or what kind of storage one would need. RWS has scraped some threads in the past for posterity - they displayed a little broken without all the custom CSS and junk but they were readable archives).

The problem. I haven't gotten any response out of HQ since I posted this thread in May. It's pretty disheartening to have made this announcement, made a bunch of promises (it'll only cost you half your history!), and then it's see you next Tuesday. Even if they were amenable to handing over the pre-2007 db, they're not available to tell me. Maybe a scrape is the way, or maybe everything's just going to stay as it is.

In the meantime, we need to make some more moves with the few tools we have at our disposal. General, FP&T, CA&P are dead as doornails, and E&A is barely ticking over. We tried not to get too aggressive with these in the first merge, but I think it's time to bring them together in one discussion area.
Mickstah +

"Science needs to be applied to beds"

Mickstah's Avatar
Joined
Jun '02
Times thanked
<
Thanks: 3,850
Thanked 4,159 Times in 1,819 Posts
Posts
25,889
Katie seems to be occasionally active here. Might be worth pinging her on Twitter to see if she's able to comment?
ಥ_ಥ - - - - - - - - - - - -ಠ_

"She was like a candle in the wind... unreliable."
Dean Learner
Trase Love:
O Mick you are so cool
More cooler than a dolphin
Let's hug me


Dr Rudi +

Howzit!

Dr Rudi's Avatar
Joined
Jan '14
Times thanked
<
Thanks: 1,176
Thanked 1,843 Times in 721 Posts
Posts
2,086
Potentially divisive suggestion here.. but how about hiding user joined dates from the main thread view as a way to help new users not feel put off posting by seeing others who've been posting here so much longer?
Bye now!
Mickstah +

"Science needs to be applied to beds"

Mickstah's Avatar
Joined
Jun '02
Times thanked
<
Thanks: 3,850
Thanked 4,159 Times in 1,819 Posts
Posts
25,889
What are you talking about? I've only been posting for 13 years. Its no big deal…
ಥ_ಥ - - - - - - - - - - - -ಠ_

"She was like a candle in the wind... unreliable."
Dean Learner
Trase Love:
O Mick you are so cool
More cooler than a dolphin
Let's hug me


NismoR31 +

NismoR31's Avatar
Joined
Feb '02
Times thanked
<
Thanks: 17,833
Thanked 10,810 Times in 4,660 Posts
Posts
34,917
boo.
ABitPatchy +

( ͡° ͜ʖ ͡°)

ABitPatchy's Avatar
Joined
Aug '06
Times thanked
<
Thanks: 2,831
Thanked 3,550 Times in 1,587 Posts
Posts
6,680

Quote:

Originally Posted by Mickstah View Post

Katie seems to be occasionally active here. Might be worth pinging her on Twitter to see if she's able to comment?

She's busy writing top 10 lists.
CAT API
Page 2 of 2
Reply

« Previous Thread Next Thread »

Posting Rules

+
    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts