Migration of SVN revision history

If you have any 'mechanical' forge related issues/suggestions, pop them in here.

Moderator: RussW

Forum rules
Locked
User avatar
Geraint
Joomla! Guru
Joomla! Guru
Posts: 561
Joined: Fri Aug 19, 2005 5:23 pm
Location: Gogledd Cymru

Migration of SVN revision history

Post by Geraint » Fri Apr 06, 2007 12:40 pm

Is there a way to migrate a repository from the old to new forge? The JEvents project has a lot of development history we don't really want to loose  :-\.

I;m not a SVN expert but I have been experimenting with svnsync to create a mirror on my own development machine - if this worked then I'd look at doing the same for the new forge.

I can initialise the synchronisation ok but I can't get the synchronization to work.    I authenticate on both servers ok but then get an error

svnsync: REPORT request failed on 'http://scm.joomla.org/svn/repos/jevents'
svnsync: The requested report is unknown.


Should this approach be possible/is there an alternative way to migrate the repository?

Thanks

Geraint

 
User avatar
Websmurf
Joomla! Hero
Joomla! Hero
Posts: 2230
Joined: Fri Aug 19, 2005 2:23 pm
Location: The Netherlands
Contact:

Re: Migration of SVN revision history

Post by Websmurf » Fri Apr 06, 2007 3:28 pm

Yeah, i'm wondering the same thing.
I tried to do a relocate, but that didn't work either.

Getting an error about the uuid's not matching
Adam van Dongen - Developer

- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl

friesengeist
Joomla! Guru
Joomla! Guru
Posts: 842
Joined: Sat Sep 10, 2005 10:31 pm

Re: Migration of SVN revision history

Post by friesengeist » Fri Apr 06, 2007 4:22 pm

Geraint wrote: Is there a way to migrate a repository from the old to new forge? The JEvents project has a lot of development history we don't really want to loose  :-\.

I;m not a SVN expert but I have been experimenting with svnsync to create a mirror on my own development machine - if this worked then I'd look at doing the same for the new forge.

I can initialise the synchronisation ok but I can't get the synchronization to work.    I authenticate on both servers ok but then get an error

svnsync: REPORT request failed on 'http://scm.joomla.org/svn/repos/jevents'
svnsync: The requested report is unknown.
SvnSync is not working because it needs a method (REPORT) which was added in SVN 1.4, but the old forge is running 1.2.3 IIRC.

The migration of the Joomla! SVN repository was done with SVM. I think it's not possible for you to upload your existing repository to the new forge though, even if you are able to retrieve it to a local repository. The reason is that it should not be possible for anyone to modify revision-properties (without specifically enabling some pre-revprop-change-hooks), which hold information about the person who committed a revision, when it was done etc. So if you mirror you old repo, you would loose all the commit times and commiter's names.

I'm not 100% sure about this, but I think that if you answered the mail every project admin received when the new forge was annouced, your old SVN will be transfered to your new project sometime. I'm interested in this as well, maybe someone from the JoomlaCode team can enlighten us?
Websmurf wrote: Yeah, i'm wondering the same thing.
I tried to do a relocate, but that didn't work either.

Getting an error about the uuid's not matching
I think you are trying to accomplish something different - switching your SVN checkout from the old forge to the new one? If so, here's the reason it does not work.
We may not be able to control the wind, but we can always adjust our sails

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Fri Apr 06, 2007 5:03 pm

Matt has been busy with preparing the migration. During the weekend, I will know how far he has come so far.

User avatar
ianmac
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 4784
Joined: Sat Sep 24, 2005 11:01 pm
Location: Toronto, Canada

Re: Migration of SVN revision history

Post by ianmac » Sun Apr 08, 2007 3:38 am

friesengeist wrote:

I think you are trying to accomplish something different - switching your SVN checkout from the old forge to the new one? If so, here's the reason it does not work.
Actually, for straight copies, it might be possible to import in to the new SVN in such a way that the switch statement will work.  I don't know for sure if this will be possible though...  In some circumstances it is possible to force a UUID on a repository, making this possible.

This was not possible on the Joomla! SVN because the structure of the repository was changed.  There are now three main directories - development, documentation, and q&t.  What used to be the root of the old repository now resides in the development directory.

We'll have to wait and see how Mat does it and then find out how it goes.

Ian

User avatar
Geraint
Joomla! Guru
Joomla! Guru
Posts: 561
Joined: Fri Aug 19, 2005 5:23 pm
Location: Gogledd Cymru

Re: Migration of SVN revision history

Post by Geraint » Thu Apr 19, 2007 5:55 pm

Any updates on if it will be possible to do a SVN migration or at least a "svnadmin dump"?

Geraint

User avatar
Jonah
Joomla! Explorer
Joomla! Explorer
Posts: 299
Joined: Thu Aug 18, 2005 6:02 pm
Contact:

Re: Migration of SVN revision history

Post by Jonah » Fri Apr 20, 2007 1:47 pm

bad news:
lamad wrote: A question about migration and svn, will the whole repository be migrated or are you only going to migrate the latest revision?  The main point of this question is if I will have access to past revisions after svn has been migrated.
Tonie wrote: Most likely it's the last revision only. The version of SVN (<1.4) used on the Forge server doesn't allow exporting the complete SVN history.
What are our options people?  Beg VA for a tarball of the entire repository?  Script something that will co and cm each individual revision so as to rebuild history?

Of course this is assuming that the forge svn comes back up.  As I'm sure some of you have noticed, while the forge website is up again the svn is still down.
trinitywebhosting.com - Free Joomla! websites for churches and non-profits.

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Fri Apr 20, 2007 1:51 pm

I'll ask Matt as well, and I didn't do SVN was down on Forge. Will look into that.

User avatar
ianmac
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 4784
Joined: Sat Sep 24, 2005 11:01 pm
Location: Toronto, Canada

Re: Migration of SVN revision history

Post by ianmac » Fri Apr 20, 2007 2:51 pm

Tonie wrote: Most likely it's the last revision only. The version of SVN (<1.4) used on the Forge server doesn't allow exporting the complete SVN history.
Hmmm...  there is a mistake somewhere...  According to the svnbook for 1.2, look at:
http://svnbook.red-bean.com/en/1.2/svn. ... k.svnadmin

they still had svnadmin dump which allows you to dump the whole repository.

Also, you will notice that the Joomla! repository on the new forge has all the history from the old forge.  And no, nobody entered 6000 or so by hand :)

The question is how well it can be scripted to move 833 projects over (though there will be some that won't need this).

From what I understand, we did manage to get a dump of the entire old forge, so we have at least something to work from.  I trust  and am confident that when Matt gets his script working and tested we will be able to move the SVN history over.

Ian

User avatar
Websmurf
Joomla! Hero
Joomla! Hero
Posts: 2230
Joined: Fri Aug 19, 2005 2:23 pm
Location: The Netherlands
Contact:

Re: Migration of SVN revision history

Post by Websmurf » Fri Apr 20, 2007 2:54 pm

I do hope the latest dump will be used.
I have committed stuff to the old repository since I got the mail about the move, would hate to loose those commits..
Adam van Dongen - Developer

- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl

motin
Joomla! Apprentice
Joomla! Apprentice
Posts: 16
Joined: Wed Feb 01, 2006 5:47 am
Contact:

Re: Migration of SVN revision history

Post by motin » Mon Apr 23, 2007 1:06 pm

I answered the e-mail one month ago but have not received any feedback.

Hope they will move the repositories soon!

I have done a lot of subversion migrations myself, having moved two repositories between 3 servers now and updating 6 working copies thereafter. Give me some credentials and I will help out moving repos.

Couldn't find the contact details of the Forge team however - have you had any better luck?

User avatar
Websmurf
Joomla! Hero
Joomla! Hero
Posts: 2230
Joined: Fri Aug 19, 2005 2:23 pm
Location: The Netherlands
Contact:

Re: Migration of SVN revision history

Post by Websmurf » Thu May 10, 2007 8:41 pm

Any update on the migration?
Adam van Dongen - Developer

- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Thu May 10, 2007 11:45 pm

Now that we have the backup of SVN, Sam can work ro incorporate it into current Joomlacode. More information when we set the migration steps that have to be taken for SVN. Hopefully he will have it done next week.

User avatar
Geraint
Joomla! Guru
Joomla! Guru
Posts: 561
Joined: Fri Aug 19, 2005 5:23 pm
Location: Gogledd Cymru

Re: Migration of SVN revision history

Post by Geraint » Fri May 18, 2007 7:14 am

Hi Tonie

Any update on this?  I know that the big meeting in SF has taken a lot of your time since the last update so appreciate that the opportunity to do anything about this has been limited.

The reason I ask is that development on JEvents (and other projects I suspect) has completely stopped for a couple of months now while we wait for the migration to occur.  With Joomla1.5beta2 out this is not very helpful timing  :(

Should we just give up on the past and start again with a new snapshot?

Geraint

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Fri May 18, 2007 9:04 am

I'll ask Sam to the Jevents project first, since you don't already have an SVN repository, this should be quite simple.

User avatar
Websmurf
Joomla! Hero
Joomla! Hero
Posts: 2230
Joined: Fri Aug 19, 2005 2:23 pm
Location: The Netherlands
Contact:

Re: Migration of SVN revision history

Post by Websmurf » Fri May 18, 2007 9:27 am

is there a timeframe known for the other projects too?
Adam van Dongen - Developer

- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Fri May 18, 2007 9:33 am

Ca depends. Most likely the following will be done (of technically possible). Have to talk to Sam about that:

* All projects devs that indicated that it doesn't have to be migrated, are not going to be migrated.
* All projects that have no SVN on Joomlacode now will have their SVN migrated straight to their project.
* If a project already has a working SVN, I hope that we can modify the current SVN and add a directory in the current repository so we don't accidentally merge all kind of things.

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Fri May 18, 2007 12:43 pm

Geraint, can you try and take a look at SVN in the Jevents project? You are hereby promoted to the guinea pig, the first project (besides Joomla) that had its SVN migrated. There is a difference now, there's only one repository instead of multiple (GForge vs Forge functionality).

Can you reply here if all is ok?

User avatar
Geraint
Joomla! Guru
Joomla! Guru
Posts: 561
Joined: Fri Aug 19, 2005 5:23 pm
Location: Gogledd Cymru

Re: Migration of SVN revision history

Post by Geraint » Fri May 18, 2007 1:02 pm

Having one repository is great - we rearranged the tree at one point and got stuck with the orphans from the past.  Having one repository will make all this a LOT easier to manage.

1. Looks great in the web browser via the project pages
2. checkout works fine
3. commit worked fine

Haven't tried a switch yet - but I can't see it being a problem.

Thanks a lot - great work

Geraint

User avatar
Geraint
Joomla! Guru
Joomla! Guru
Posts: 561
Joined: Fri Aug 19, 2005 5:23 pm
Location: Gogledd Cymru

Re: Migration of SVN revision history

Post by Geraint » Fri May 18, 2007 1:27 pm

There was a tiny problem when relocating an existing checked working copy but I could do a fresh checkout and copy and overwrite the checked out files from my working copy.

Thanks again

Geraint

p.s. Note important but the revision numbers were bumped up by about 5-6, would this make any sense?

User avatar
Tonie
Joomla! Master
Joomla! Master
Posts: 16584
Joined: Thu Aug 18, 2005 7:13 am

Re: Migration of SVN revision history

Post by Tonie » Fri May 18, 2007 1:33 pm

I'll ask Sam about the revision numbers. I noticed the same with the relocating of SVN code with Joomla! SVN initially, I will add it to my notes so I can mention it in the email to devs again.

Cool, looks like we can go on and finish this migration. It's pretty safe to say we're getting tired of it ;).

User avatar
Jonah
Joomla! Explorer
Joomla! Explorer
Posts: 299
Joined: Thu Aug 18, 2005 6:02 pm
Contact:

Re: Migration of SVN revision history

Post by Jonah » Fri May 18, 2007 6:57 pm

Geraint wrote: The reason I ask is that development on JEvents (and other projects I suspect) has completely stopped for a couple of months now while we wait for the migration to occur.  With Joomla1.5beta2 out this is not very helpful timing  :(
Ack that.  RSGallery2 also stopped for over a month.  Eventually we just took a snapshot and imported it.  However, we need history to merge changes from trunk to a branch.  So our J1.5 branch continues to languish.  As soon as we have history we can work on the J1.5 version again.
trinitywebhosting.com - Free Joomla! websites for churches and non-profits.

fmadec
Joomla! Fledgling
Joomla! Fledgling
Posts: 3
Joined: Mon Apr 23, 2007 9:24 am

Re: Migration of SVN revision history

Post by fmadec » Mon May 21, 2007 8:03 am

Any updates on svn migration?

I understand that you have a lot of work but a timeframe will be great.

thanks

Fab

User avatar
ianmac
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 4784
Joined: Sat Sep 24, 2005 11:01 pm
Location: Toronto, Canada

Re: Migration of SVN revision history

Post by ianmac » Tue May 22, 2007 3:33 pm

Tonie wrote: I'll ask Sam about the revision numbers. I noticed the same with the relocating of SVN code with Joomla! SVN initially, I will add it to my notes so I can mention it in the email to devs again.

Cool, looks like we can go on and finish this migration. It's pretty safe to say we're getting tired of it ;).
How far off were the Joomla! SVN revision numbers?  AFAIK, they should have been off by 1 because the repo was restructured.  The root is not development, documentation and qt, whereas before everything under development was in the root.

Just in case this had been forgotten...

Ian

 

Locked

Return to “JoomlaCode.org”