Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Relax and enjoy The Lounge. For all Non-Joomla! topics or ones that don't fit anywhere else. Normal forum rules apply.
Post Reply
User avatar
fatbear
Joomla! Apprentice
Joomla! Apprentice
Posts: 41
Joined: Tue Mar 28, 2006 4:09 pm
Location: Raleigh, NC
Contact:

Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by fatbear » Sun Aug 12, 2018 12:00 pm

What is the approach that is recommended for dev/test/prod staged deployments that are able to preserve dynamic data in the prod environment (such as forum data, e-commerce data, etc.)?

I imagine that the process is essentially:
  • Backup the prod site (site.com) with tools such as Akeeba Backup as dev.site.com
  • Make structural changes in dev, update extensions, update articles, etc.
  • Before pushing to test, re-get current database data from prod to dev
    • This is the part I don't know how to do. I imagine it is necessary to determine which tables need to be sync'd based on various factors: content type, extension type, etc.
  • Backup the dev site (dev.site.com) and deploy that as a test site (test.site.com).
  • Get stakeholders to approve the changes in the fully functional test site
  • Upon approval, put prod site in maintenance read-only mode and re-get current database data from prod to test
  • Backup the test site (test.site.com) and deploy that as the new prod site (site.com)
Can anyone offer any comments? Am I on track here? Are there any built-in features to assist in database import/export in a way that preserves dynamic prod content? Are there any extensions to do this?
Steve Amerige
Server Science Incorporated
Reusable Software Development, Open Source Development: https://eggsh.com

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 36145
Joined: Sat Apr 05, 2008 9:58 pm

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by Webdongle » Sun Aug 12, 2018 2:47 pm

afaik it's not possible with Joomla
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein

User avatar
brian
Joomla! Master
Joomla! Master
Posts: 11724
Joined: Fri Aug 12, 2005 7:19 am
Location: Leeds, UK
Contact:

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by brian » Sun Aug 12, 2018 4:05 pm

This is all possible with the superb service at https://deevop.com/
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 36145
Joined: Sat Apr 05, 2008 9:58 pm

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by Webdongle » Sun Aug 12, 2018 6:40 pm

But does that update the developed site to the live site without losing changes made(during development) to the live site?

The request by the OP "to assist in database import/export in a way that preserves dynamic prod content?" ... can not be done.
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein

User avatar
brian
Joomla! Master
Joomla! Master
Posts: 11724
Joined: Fri Aug 12, 2005 7:19 am
Location: Leeds, UK
Contact:

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by brian » Sun Aug 12, 2018 6:49 pm

yes or i would not have suggested it
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 36145
Joined: Sat Apr 05, 2008 9:58 pm

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by Webdongle » Sun Aug 12, 2018 7:17 pm

That is interesting. So if the live site has Articles etc added while the development site is being developed ... then the changes in the dev site can be added to the database of the live site without wiping the additions to the live site? Wow learn something every day. Thanks Brian.
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein

User avatar
fatbear
Joomla! Apprentice
Joomla! Apprentice
Posts: 41
Joined: Tue Mar 28, 2006 4:09 pm
Location: Raleigh, NC
Contact:

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by fatbear » Sun Aug 12, 2018 7:19 pm

Hi Brian,

Thanks for the response. I am developing a corporate website. The site must be entirely hosted on my client's servers. Is your solution self-hosted? Is it a Joomla extension or other software that can be installed within the client environment without making any calls to an external website?
Steve Amerige
Server Science Incorporated
Reusable Software Development, Open Source Development: https://eggsh.com

User avatar
brian
Joomla! Master
Joomla! Master
Posts: 11724
Joined: Fri Aug 12, 2005 7:19 am
Location: Leeds, UK
Contact:

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by brian » Sun Aug 12, 2018 7:21 pm

Best to spend the time and read the site than going back and forward with questions here
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/

User avatar
fatbear
Joomla! Apprentice
Joomla! Apprentice
Posts: 41
Joined: Tue Mar 28, 2006 4:09 pm
Location: Raleigh, NC
Contact:

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by fatbear » Sun Aug 12, 2018 7:43 pm

I actually did. It did not appear that the website presented any extensions for purchase. From reading the website, I guessed that the company provides hosting and the development workflow is offered as a part of the services to their hosted clients. But, I left open that they have undocumented extensions that are available to do this task.

Do you know if they offer any such extension for sale? As any solution I adopt must be self hosted, do you know of any other solution to manage dev/test/prod development and deployment?

In any case, thanks for your reply.
Steve Amerige
Server Science Incorporated
Reusable Software Development, Open Source Development: https://eggsh.com

User avatar
sozzled
Joomla! Champion
Joomla! Champion
Posts: 6069
Joined: Sun Jul 05, 2009 3:30 am
Location: Canberra, Australia
Contact:

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by sozzled » Sun Aug 12, 2018 8:04 pm

I'm not going to get technically involved with this matter because @brian and @Webdongle will leave me for dead. I think @brian's suggestion has merit and worth investigating; as he wrote, it's something you should examine for yourself.

As far as your overall strategy was outlined, in going from developement -> test -> production, there are some scenarios that may impact your choices. Managing the change from a mature, stable production environment through to a new, "improved" environment needs stakeholder involvement, of course.

If the two environments are quite different to one another—if there are changes within the database structures, that is—then the change will be non-linear. The non-linear change, in this situation, involves structurally modifying the original (production) data—the content—when the switch-over to the changed state takes up content additions, modifications or deletions that occurred from the time the two states were in last in continuum with one another. It depends, of course, on the complexity and duration of the project life cycle.

If, for example, we were discussing a project that involved updating a website from J! 3.8.10 to J! 3.8.11 and the customer wanted a demonstration of the "new site" before committing to the work, then this could be accomplished by creating a demonstration website as a feasibility study; allow the customer (or a team of players, a "business focus group" as I've used the term myself) to play with the demo site and, once the customer is satisfied that the process is sound, the actual work could be carried out on the production site. Of course that's a facile example.

In a situation where a project runs over several weeks or months, the approach I've just mentioned may not be workable. In that situation, @brian's suggestion looks like it has the necessary version control mechanisms in place and may be a more practical way to go. Perhaps the solution lies somewhere in-between?
https://www.kuneze.com/blog
Former member of Kunena project team
If you think I’m wrong then say “I think you're wrong.” If you say “You’re wrong!”, how do you know?

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 36145
Joined: Sat Apr 05, 2008 9:58 pm

Re: Approach for Staged Development Wanted: Dev/Test/Prod with Dynamic Data

Post by Webdongle » Sun Aug 12, 2018 9:47 pm

If you simply wanted to show a development version then create a subdomain and place it there. If you want to show possible developments of the live site then ... (when the development site is agreed on) you will need to replace the live site completely. Or make the same changes in the live site.

If (while you work on the development site) the live site gets added to then ... the additions to the live site will be lost.

If the changes (made in the dev site) are with the template then you just need to copy the files to the live site. If the changes (made in the dev site) are are adding or removing extensions then ... (to keep the additions to the live site) you need to make the same changes in the live site.

Your requirement
"Backup the test site (test.site.com) and deploy that as the new prod site (site.com)"
Can not be done. The dev site can not be appended to the live site ... it must replace the live site. If the live site has no additions that's fine. But if things have been added to the live site then they will be lost.
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein


Post Reply

Return to “The Lounge”