SOLUTION to Forge Downtime

sevenhelmets · Post by **sevenhelmets** » Fri Mar 09, 2007 2:27 am

To Joomla admins/devs

I know you guys are busy and probably stressed outta your minds, so I'll keep this short and sweet. There is a lot of bickering going on about how people can't get files they need etc etc. Most of them are hosted on the Forge as we all know. Mirrors appear to be short and far between.

How about if all of this stuff was put up onto Bit Torrent? And I mean "the whole lot", the actual Joomla 1.0.1.2 etc files as well as 3rd party extensions, components, modules etc. It would mean a bit more work setting up a private tracker (I honestly don't think a public tracker would be a "safe" thing to do), but at least that way the community supports itself by it's users seeding the files, meaning minimum bandwidth expense for the project itself.

I know the cons of Bit Torrent is that some people will have absolutely no idea what it is, and possibly drive them away, but at the end of the day it's only a solution while the Forge is down. Once the Forge is back, then Bit Torrent side of things is simply an alternative way of getting the files. You might even find it'll decrease the traffic to the Forge, and therefore the bandwidth........

ChiefGoFor · Post by **ChiefGoFor** » Fri Mar 09, 2007 2:37 am

Please be patient while we fix the Forge issue. I want to stress that using BitTorrent for this is a bad idea. The reason for this is that you never REALLY know what you are downloading. If I was a bad person, I could add some code to the release and then release it on BitTorrent. Then, you would never know if it was legit or something that will give a hacker an easy way in. So, with that said, please be patient. If you are looking for Joomla 1.0.12, there is a sticky at the top of the forum with an alternative link to get it. Thanks!

sevenhelmets · Post by **sevenhelmets** » Fri Mar 09, 2007 2:53 am

ChiefGoFor wrote: The reason for this is that you never REALLY know what you are downloading. If I was a bad person, I could add some code to the release and then release it on BitTorrent. Then, you would never know if it was legit or something that will give a hacker an easy way in.

Well that's why I was suggesting a private tracker....ie one which is actually run by Joomla itself. Although I guess as you pointed out, its still no deterrent for a hacker. Then again, if someone were that determined, they'd get into anything anyway, including the Forge, regardless of security.

Jaccobb · Post by **Jaccobb** » Fri Mar 09, 2007 2:58 am

ChiefGoFor wrote: If I was a bad person, I could add some code to the release and then release it on BitTorrent.

I guess the question is then... are you a bad person?

ChiefGoFor · Post by **ChiefGoFor** » Fri Mar 09, 2007 1:41 pm

Jaccobb wrote:
ChiefGoFor wrote: If I was a bad person, I could add some code to the release and then release it on BitTorrent.
I guess the question is then... are you a bad person?

Nope, I'm not.

ianmac · Post by **ianmac** » Fri Mar 09, 2007 2:08 pm

I trust that the Forge issue is being addressed in all ways possible. I am willing to wait patiently while it is addressed.

But I wonder if BitTorrent might be an idea for down the road in the future?

The issue about verifiability is an important one. Once the forge issues get resolved, is it worth looking into BitTorrent as a way to reduce the load off of the main servers? Then we would only have to host some sort of public key/private key/certificate or MD5 sum that people can download off of the forge which will ensure that the packages are the right packages.

Though I wonder what the biggest load on the forge is? SVN updates, web access or downloads?

Ian

brian · Post by **brian** » Fri Mar 09, 2007 2:13 pm

To be perfectly honest I dont believe that either the number of consecutive users or the volume of traffic is the issue. Rather it is just that SFEE cannot cope with it.

I trust that the S&I team are resolving this with a more rbust solution backed by a host with experience.

ChiefGoFor · Post by **ChiefGoFor** » Fri Mar 09, 2007 2:13 pm

I really want to try to crush the whole BitTorrent idea. It is wrong on so many levels. People do not really looks at MD5 hashes, people do not look at keys, most people do a search, then download. The last thing this community needs is to have the stereotype that if you download Joomla!, you do not know what virus you will download too. Hosting providers will stop allowing it to be installed on their servers and so on. Please, everyone, understand that.

ianmac · Post by **ianmac** » Fri Mar 09, 2007 2:22 pm

I really want to try to crush the whole BitTorrent idea. It is wrong on so many levels. People do not really looks at MD5 hashes, people do not look at keys, most people do a search, then download. The last thing this community needs is to have the stereotype that if you download Joomla!, you do not know what virus you will download too. Hosting providers will stop allowing it to be installed on their servers and so on. Please, everyone, understand that.

Fair enough... I guess it isn't really that it couldn't work, but that it wouldn't work because most people are too lazy to check the hashes and keys

Ian

boylan · Post by **boylan** » Sun Mar 11, 2007 6:15 pm

Doesn't the Bit Torrent protocol do hashing automatically? I mean if Joomla controls the tracker - the files linked to in the tracker cannot be spoofed. It isnt like Kazaa - where anyone can call any file any name.

Sure, someone else could set up another tracker that linked to a bad version of Joomla, but someone could much more easily offer up a bad version for download.

If the main worry is that people could swap good versions for bad, if Joomla controlled the tracker - that would be impossible. There are other reasons not to use BT, but this should not be one of them.

http://en.wikipedia.org/wiki/Bit_torren ... g_torrents

Or am I totally incorrect? (The possibility exists)

Another reference

ChiefGoFor · Post by **ChiefGoFor** » Mon Mar 12, 2007 12:10 am

http://forum.joomla.org/index.php/topic,148843.0.html

We are working on a new forge to fix all these problems. Therefore a torrent is not necessary.

quayph · Post by **quayph** » Tue Mar 13, 2007 7:54 am

Whilst I do agree that Bittorrent may not be the solution, for the same security / credibility reasons that you mention, I must say that I do not agree when you say that an alternative to Joomla forge is "not necessary".

That is like saying that the seatbelts in your car are not necessary because you never crash.

The fact that the forge is down proves that some sort of redundancy is necessary. Any new system will eventually have some sort of problem too. (unless the joomla codebase and userbase stops growing, the servers never have any mechanical breakdowns, the new system is 100% bug free from the word go, etc...)

Thankyou to everyone who has reacted quickly to give us alternative download sites for the current Joomla distribution. Unfortunately, the real beauty of the Joomla forge is all the 3rd party extensions that are still unavailable.

Once Joomla forge is back up and running the site administration team should give some serious thought to a system of mirroring.

cyburdine · Post by **cyburdine** » Tue Mar 13, 2007 7:21 pm

I write this with all due respect, but as a long time user/supporter of Joomla I feel like I need to point out that the admin responses I'm reading here are counter to the spirit in which Joomla was born. Joomla is all about openness, open source, open community. But what I am reading is the opposite... The jist is "yeah there's a problem, but we're working on it... it's going to be great, trust us. no we don't want input, just trust us."

To simply shoot down an Idea because you think people are inherently dumb, doesn' make a single point of failure solution any better. Please share with us what the new forge solution is so that the community can evaluate it and understand how much better it's going to be, perhaps even add input. That's open source mattering.

just my .02 on this subject...

Sincerely,
cyburdine

who would I talk to about offering up my systems as a public backup mirror?

mpgonline · Post by **mpgonline** » Tue Mar 13, 2007 7:37 pm

I would love to use some components if I could get them from forge.joomla.org. Many be all the extenstions should be moved since it never seems to work!!

louis.landry · Post by **louis.landry** » Tue Mar 13, 2007 8:06 pm

cyburdine wrote: I write this with all due respect, but as a long time user/supporter of Joomla I feel like I need to point out that the admin responses I'm reading here are counter to the spirit in which Joomla was born. Joomla is all about openness, open source, open community. But what I am reading is the opposite... The jist is "yeah there's a problem, but we're working on it... it's going to be great, trust us. no we don't want input, just trust us."

To simply shoot down an Idea because you think people are inherently dumb, doesn' make a single point of failure solution any better. Please share with us what the new forge solution is so that the community can evaluate it and understand how much better it's going to be, perhaps even add input. That's open source mattering.

just my .02 on this subject...

Sincerely,
cyburdine

who would I talk to about offering up my systems as a public backup mirror?

Your $.02 are noted ... as are everyone else in this thread's $.02.

Have you considered that perhaps there is a reason that we aren't speaking more openly about all of this? Your assertion that we are shooting ideas because we think people are dumb is beyond belief to me. We are an open group that takes in advice from people on a daily basis on almost everything we do. Open source doesn't necessarily mean that you get to be intimately involved in every aspect of a project or everything that surrounds it. It means different things to different people. Your meaning tends to point with the broadest stroke possible and that is your prerogative.

The S&I working group is responsible for Sites and Infrastructure ... thus the name

I would suggest speaking to Brad or Tonie as they are Coordinators of that working group.

Now I'm going back to working on the resource to help us all ... thanks for your patience

Louis

schteff · Post by **schteff** » Tue Mar 13, 2007 8:14 pm

when is forge comming back online?

I would really like to use the resources available at that server.

ChiefGoFor · Post by **ChiefGoFor** » Tue Mar 13, 2007 11:12 pm

@cybudine - Your points are noted. Thank you!

@schteff - Please see this announcement: http://www.joomla.org/content/view/2936/1/
I know it is tough to wait, but there are many people putting in long hours trying to get this thing up and running. I have to say that I do not think anyone will be disappointed with what is replacing the forge.

Post by **JoeJoomla** » Wed Mar 14, 2007 3:04 pm

boylan wrote:There are other reasons not to use BT

I think BitTorrent has a long way to go before I would include it in production or business processes. I could be wrong but I haven't been impressed with BitTorrent to date. Mind you, I use a Macintosh computer, Mac BitTorrent clients may have something to do with it. No reputable company I know uses BitTorrent for their products. Also any time in the past that I have tried BitTorrent I have been underwhelmed.

I think the real solution lies in providing robust proven technologies that are properly scaled to meet the demands. I wouldn't trust such a large undertaking of valuable data assets to BitTorrent unless a bullet proof model proof of concept had been done. I don't blame the decision makers for nixing this idea. It doesn't mean that it shouldn't be proposed, every idea should be brought forward and considered. I like the concept of BitTorrent, it's the practice that I am not confident in yet.

sevenhelmets · Post by **sevenhelmets** » Thu Mar 15, 2007 1:54 am

This is pretty much a closed issue from J! admins/mods position, particularly with the Forge back up and running, but I thought I'd just post in a defense for the torrent idea. I'm not doing this particularly because "it's my idea" - I respect the decision the mods/admins have and I'm only here to offer any help I can back into the community that has in turn helped me out so much. However I do feel I have a pretty good view on things regarding torrents, and hopefully some of this knowledge can be used in some way or another to help out the community.

Basically my idea centers around the idea that torrents would be on a private tracker. A lot of people may have bad memories associated with downloading false or fake files that were put up on public trackers "back in the day" like Suprnova or Piratebay etc etc. These sites are high risk because pretty much anyone can upload anything, including as Chief mentioned, things like virus or trojan ridden software.

You don't have these issues with a private tracker. Creating the torrents can be restricted to any number of admins and mods, thereby ensuring the security of the files available for download. At the moment I would presume this is the same as the Forge anyway - if a 3rd party developer of a component wishes to upload their product, they have to send it to one of the mods/admins, who will test it and then upload it for them. If this is not the case, and the developer uploads it directly to the Forge, then the security level is simply the same as if you shared via torrent on a public tracker, and that argument of increased security is void.

Obviously the downside to this is that it requires the mods/admin's time to maintain the whole thing, including uploading of new torrents. This can be quite an undertaking considering the number of beta releases and patches that keep on getting issued by the developers. Further discussion would have to be made on this point. A probable solution would be that seeing as these releases would be small(er) in size, they would not need to be made available via torrent. This restricts the amount of torrents uploaded to "stable" releases, which are probably further between each other. Take for example hosting a torrent of the current core Joomla 1.0.12 file. This was released at the end of last year, so it would've taken a few minutes of someone's time to upload, and you could forget about it until a new version is released.

At the end of the day the main idea is that it isn't a "be all and end all" solution. It is simply an alternative to the Forge, if it's required. With the new Forge in place this idea will probably get scrapped even further into obscurity, however as I said, this idea is only as a "precaution" and further measure of supporting the community. I have huge respect for the mods/admins/developers and don't wish to discredit all their hard work put into bringing the Forge back up to speed.

gwildgoose · Post by **gwildgoose** » Thu Mar 15, 2007 6:48 pm

of course you all realize that I am just having a little fun with this.....

So when will the forge be up and running?

Keep up the good work guys we really do appreciate it.

ChiefGoFor · Post by **ChiefGoFor** » Thu Mar 15, 2007 6:50 pm

Um... now. http://www.joomla.org/content/view/2965/1/

lejovchina · Post by **lejovchina** » Sun Mar 25, 2007 11:13 pm

Down again... just when i need it the most.
Damn!

# date && ping forge.joomla.org
Mon Mar 26 01:10:59 CEST 2007
ping: unknown host forge.joomla.org

Ljv.

brian · Post by **brian** » Sun Mar 25, 2007 11:14 pm

pleaseread http://www.joomla.org

The forge has been replaced with joomlacode.org

lejovchina · Post by **lejovchina** » Sun Mar 25, 2007 11:22 pm

I am aware of that change... Only complaining because i was in the middle of getting the module for Attend Events, which is still not in the new forge, as far as i could search.

Ljv.

brian · Post by **brian** » Sun Mar 25, 2007 11:26 pm

is the role of the individual developer to move

lejovchina · Post by **lejovchina** » Sun Mar 25, 2007 11:58 pm

In my opinion, it should be the role of the forge admins to give each dev more time and keep the damn repository for backwards reference. Another brilliant decision. Downhill, babe! Eeeeee-hhhaaaaa! $:-\$

Ljv.

ianmac · Post by **ianmac** » Mon Mar 26, 2007 12:03 am

Wow, that was awfully disrespectful! Here we have volunteer people putting countless amounts of effort into building and developing a secure platform upon which we can do Open Source Joomla! development, and you are criticizing. As an FYI, the old Forge does still work. There will be downtime but please recognize that the server upon which it is hosted is out of our control.

Your sarcasm is completely unhelpful and demonstrates a lack of understanding of the situation and a lack of respect for people who have worked really hard to find a solution to the challenge that we as a community are facing.

The forge admins will give plenty of opportunity for developers to transfer their sites. We also have negotiated a complete data dump so that in the worst possible case we have everything to reconstruct what we need.

Please be understanding that much of the situation with the old forge is beyond our control and people are doing their best to make things work.

Thanks,
Ian

deltina · Post by **deltina** » Mon Mar 26, 2007 12:19 am

So am i understanding this correctly: that until a developer updates his or her link, there is no way to download an extention?

I can see why people are getting cranky; yet I also know how much work y'all put into this.

I hate the thought that I may now need to recommend a completely different CMS to my present client, though, since I will not be able to integrate a forum for them now.

This is very crippling...tragic...

ianmac · Post by **ianmac** » Mon Mar 26, 2007 12:43 am

No, you are not understanding correctly.

Please read the quote:

As an FYI, the old Forge does still work. There will be downtime but please recognize that the server upon which it is hosted is out of our control.

I guess you haven't tried, but my recent attempt to access the old forge did not cause me any problems.

All I was trying to say was that there may be intermittent downtime in meantime, but all in all the old forge should work until we get everything transferred over.

The sky is not falling, I don't see this situation as being crippling. If the old forge doesn't work for you, give it some time and try again. Since we have moved over this has reduced a good deal of the load and I have not had significant problems accessing the old forge. There are still some maintenance tasks involved in moving over, but there should be minimal impact.

Ian

brian · Post by **brian** » Mon Mar 26, 2007 12:49 am

oops

ian can you try right now. it looks like forge.joomla/org has been removed from dns

http://www.dnsstuff.com/tools/lookup.ch ... org&type=A

Searching for forge.joomla.org A record at g.root-servers.net [192.112.36.4]: Got referral to TLD4.ULTRADNS.org. (zone: org.) [took 37 ms]
Searching for forge.joomla.org A record at TLD4.ULTRADNS.org. [199.7.67.1]: Got referral to ns4.mydyndns.org. (zone: joomla.org.) [took 8 ms]
Searching for forge.joomla.org A record at ns4.mydyndns.org. [213.155.150.206]: Reports that no A records exist. [took 82 ms]

Response:
No A records exist for forge.joomla.org, and forge.joomla.org does not exist. [Neg TTL=1800 seconds]

Details:
ns4.mydyndns.org. (an authoritative nameserver for joomla.org.) says that there are no A records for forge.joomla.org, and that the hostname forge.joomla.org does not exist.
The E-mail address in charge of the joomla.org. zone is: [email protected].

The Joomla! Forum™

SOLUTION to Forge Downtime

SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime

Re: SOLUTION to Forge Downtime