031 - Create a unit test plan for the Module Manager

Google's Highly Open Participation Program tries to get young students into Open Source and Joomla! specifically. Everyone is welcome, there are not limits. You can be a coder, documenter, tester, translator to help out. Jump in and start helping!
Locked
Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

031 - Create a unit test plan for the Module Manager

Post by Draicone » Wed Jan 09, 2008 1:09 pm

Hi guys!

I'm doing a unit test plan for the module manager -- issue #31. I've had a bit of experience with it, having built a module for my last issue (#43, changelog parser), but it'll still take a bit of exploring.

I was thinking of taking a sequential procedure approach to documenting tests -- e.g. a section for creating a new module, and within that a test for clicking the 'New' button, a test for selecting a module radio button and clicking the 'Next' button etc. It seemed to make more sense as a series of related tests like this would be conducted one after the other anyway; Rahul's unit test plan was very detailed but got a little complicated with all the structure, and I don't intend to have such a deep hierarchy in the plan. Tests documented in an order suitable for being carried out, and structured in a way that reflects this, should save some extra work making sense of the unit test plan for testers.

Task:
Create a unit test plan that identifies at least 25 specific tests and expected results that could be used to validate Joomla! v 1.5 development. The test plan should specifically list each action that would be taken by a tester and the expected results. A tester should be able to use the document to verify the accuracy of the Joomla! development.
Please let me know if there's anything I should watch out for or some interesting things that can (intentionally) be achieved with the module manager.

I'm looking to have some sort of a draft available tomorrow to get some feedback on this new structure I'm trying out; please subscribe to this topic and watch your inbox if you're interested in having a look and posting some feedback.
Last edited by AmyStephen on Mon Feb 04, 2008 4:19 am, edited 1 time in total.

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Fri Jan 11, 2008 9:39 am

Okay, time to break out of the box a little... (might want a developer to have a read through this, it's more a technical issue than a policy thing)

I did some research into traditional unit testing, and the approach Joomla was taking, and it all seems like it could do with a little more standardization and organization. Unit test plans are being submitted in highly varied formats, and the testing reports submitted are about the same. What we need is a standard format to record unit tests, so that in future we can develop a (very basic) unit testing system for testers to submit reports through. We can then archive reports, automatically notify the relevant developers when a test fails and do some creative things with the testing plans e.g. searching and sorting of tests.

To this end, I've decided to submit my test plan in XML. I know the task wants an ODF file (.odt?), so I'll process the XML file into HTML (although my XSLT isn't too great, so I might have to do it manually in PHP, and I can then create a library for this future unit testing system in the process). The HTML can be copy-and-pasted into OpenOffice (when I find a machine that can support it, that is).

XML solves the category problem I was discussing earlier, gives us a lot more flexibility in how we manage the unit testing for Joomla -- we could probably even hook it into the bug tracker. We also have a very useful file format -- an interesting GHOP task would be developing this unit testing system to parse the XML files and produce some useful output for testers.

I've created a couple of XML namespaces for this new system, the XML-based language of which I'm going to call JTML: Joomla Testing Markup Language. The default namespace is used for everything except the tests themselves, which are namespaced under 'jtest' e.g. . The specification is very loose at the moment, I'll submit my draft and we can always change it later. I'll put in a spec version attribute for now, just in case.

This is all a little ambitious -- possibly over ambitious -- but it could really streamine the testing process, make preparing unit testing plans much easier (and less reliant on having a copy of Open Office, helpful e.g. for me) and give us a more efficient way of tracking results of unit tests -- e.g. through (as yet non-existent) reporting interfaces.

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Jan 13, 2008 12:25 pm

Here's a sample: http://pastebin.ca/853252. Highly parseable :)

User avatar
mcsmom
Joomla! Exemplar
Joomla! Exemplar
Posts: 7897
Joined: Thu Aug 18, 2005 8:43 pm
Location: New York
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by mcsmom » Fri Jan 18, 2008 10:07 pm

Cool Daricone,

If you go to the Joomla! Summer of Code forum you will find lots of information on formal unit testing approaches that are being worked on. You might find them helpful. Meanwhile, I'll try to find someone to look at what you're doing. :)
So we must fix our vision not merely on the negative expulsion of war, but upon the positive affirmation of peace. MLK 1964.
http://officialjoomlabook.com Get it at http://www.joomla.org/joomla-press-official-books.html Buy a book, support Joomla!.

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Jan 20, 2008 2:38 am

I've finished 25 tests in this XML format and pasted them at http://pastebin.ca/865837. Please take a look; with the syntax highlighting, it's easy to see the actual content of the test plan outside of the code, and it's very self-explanatory. I'll convert it to HTML soon and I'll try to download OO.org and submit the ODT version as well. The XML file is also attached to this post.

(By the way, mcsmom -- I believe this is unrelated to the Joomla unit testing discussed at the GSOC forums as that unit testing is related to code subprocedures, whereas this is the functioning of the application from a user's perspective.)

From here on, I was hoping to further pursue this concept of a dedicated XML-based file format for unit tests. To this end, I kindly request that upon submission of the 25 (or 50) tests before the task deadline (in two days, I think), I please be granted a two week grace period to develop a system to interpret and display this file format so that it is available in a usable form and can be converted programmatically. Alternatively, I kindly request a new task please be created with a title along the lines of "Develop a PHP script to interpret and parse the JTML template language and display the contents of a unit test plan" so that I may complete this task within its scope and deadlines, then move onto that task and develop the system.

Given that user-based unit testing is essential to effective release testing, I feel that the creation of a single system with a single format for documenting unit test plans would be a fantastic addition to the release testing and would also greatly simplify the process by which tests from this unit test plan and others are carried out. A unified reporting interface would significantly reduce the amount of time required to construct and report on the results of unit tests carried out.
You do not have the required permissions to view the files attached to this post.

AmyStephen
Joomla! Champion
Joomla! Champion
Posts: 7018
Joined: Wed Nov 22, 2006 3:35 pm
Location: Nebraska
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by AmyStephen » Sun Jan 20, 2008 2:53 am

You are hearby granted an extension of time to explore this world that you are able to see that others of us obviously miss. Let's see what you come up with.  Very creative.
   
     
        Select the Module Manager menu item from the Extensions Menu
        A page with a paginated table listing of site modules, as well as their order number, status, ordering, access level, position, pages, type and ID appears. The page also has a link to the page for modules for the administrator section.
     
     
       
Click on the Administrator link in the standard module manager.
A similar page listing modules for the administrator panel appears.
     
Draicone - do yourself a favor and avoid jobs where you are required to show up at 8am each day. You won't like that. It won't work when your mind takes over with interesting thinking about you are still working at 4am. You are going to need to research and explore and innovate.

You do realize you really are not doing what the task requested? I'm cool with it, but, it'll get you in trouble if the boss really needed a list of testing requirements. That is not at all critical of what you are doing. It's an observation you should be aware of and you should prepare for as you go through life.

Pretty cool stuff, dude. Way off track. But, very innovative.

Carry on - go where you need to go! Thanks for returning for another task!
Amy :)

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Jan 20, 2008 3:03 am

Hi Amy!

Thanks for the extension!

Actually, I've got three (work from home) jobs where I'm meant to show up at 8am each day, and I more or less outsource all of them ;)

I was trying to be a little thorough with the test plan, hence starting with such basic tests and documenting it in XML format. I haven't had any time to work on it over the last week or so but I'm going to aim to reach 50 tasks and cover all the more intricate areas of the module manager. I'm also downloading OO.org right now and hopefully I'll manage to install it this time, so I can properly fulfill the task requirements.

Besides, I'm just having fun here, and it's probably best that I do now as opposed to in ten years time when I'll have performance targets to meet, bills to pay and all the other sad truths of a working life.

Anyway, thanks for the extension, I look forward to coming back in a week or two's time and having something truly innovative to show for all this creativity.

AmyStephen
Joomla! Champion
Joomla! Champion
Posts: 7018
Joined: Wed Nov 22, 2006 3:35 pm
Location: Nebraska
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by AmyStephen » Sun Jan 20, 2008 3:16 am

You already have come up with something innovative.

Keep having fun - it's possible to earn a living *and* have fun, too. You just have to be creative - and trust me, you got that cornered.

Take care,
Amy :)


PS - one last thing to consider. I believe Monday is the last day to claim a new task.

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Jan 20, 2008 6:57 am

AmyStephen wrote:PS - one last thing to consider. I believe Monday is the last day to claim a new task.
Really? I thought the competition was open till Feb 3, and figured we could keep claiming new tasks and submitting entries till then...

AmyStephen
Joomla! Champion
Joomla! Champion
Posts: 7018
Joined: Wed Nov 22, 2006 3:35 pm
Location: Nebraska
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by AmyStephen » Sun Jan 20, 2008 7:23 am

That's why I mentioned this to you. I thought you might not know that for certain.

Monday - you can select your last task - then, you have until the 3rd (I believe) to finish it.

If you want another one, I think you have made excellent progress here.

Your call.

Amy :)

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Jan 20, 2008 7:25 am

Well, an extra task won't really achieve much, except maybe applying my time to an extra area of the project. Forget it. I'm going all out on this one :)

AmyStephen
Joomla! Champion
Joomla! Champion
Posts: 7018
Joined: Wed Nov 22, 2006 3:35 pm
Location: Nebraska
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by AmyStephen » Sun Jan 20, 2008 7:26 am

8)

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Mon Jan 28, 2008 11:33 am

Okay, after multiple caffeine-fueled (I asked Wilco for some rum but never got a response :() late nights hacking away at a  not-so-fantastic feat of engineering, I've built a complete PHP application that will take these JTML XML files, import them into a database and delegate testing tasks. I'm working on a system to submit reports on these tests, it shouldn't take more than a day or two. Now I just need someone to show it to :)

I'm looking for beta testers - anyone with five minutes to briefly take a look at it and give me some feedback. It's full of access control so I can't just post admin passwords here (although I'm trying to get a demo up and running). One of the GHOP people would be preferable but anyone from the Joomla community is fine.

I mentioned the format of this JTML is still unfinalised -- I've sent off emails to a few people involved in the unit testing for some input. If anyone else would like to make a suggestion [on data associated with user-based unit testing that should be included in a file format], please let me know.

AmyStephen
Joomla! Champion
Joomla! Champion
Posts: 7018
Joined: Wed Nov 22, 2006 3:35 pm
Location: Nebraska
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by AmyStephen » Tue Jan 29, 2008 5:02 am

Draicone -

Feel free to email me at AmyStephen at gmail dot com / or IM with my via Skype AmyStephen.

I am certainly interested in seeing this in action!

Thanks...Amy :)

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Feb 03, 2008 12:46 am

Alright, it's 4AM and if I don't go to sleep now I doubt I'll wake up at any reasonable hour, be late for some family event I've apparently been invited to etc. etc.. After 40+ hours of hacking away at the system, which I've dubbed JUserTest for now, the first beta is pretty much ready. It's not as refined as I hoped -- I'll do a proper spec for the XML format soon -- but it does work, and it's pretty good as it is.

Just a bit of background for those who can't be bothered reading through my excessively lengthy posts: Currently, unit test plans in GHOP are being submitted in ODT and PDF, with inconsistent formatting, different ways of expressing tests etc. I propose to create a consistent, generic XML-based file format which I call JTML. If we create this file format, we can in future plug the test plan XML files into a unit test tracking system to run organised testing efforts.

Of course, me being me, I wrote the unit test plan for the task, then created the XML-based file format, and just for the heck of it built the test tracking system as well.

The test system is written in PHP/MySQL, running on top of CodeIgniter. I would have loved to write it in J!1.5, but I simply can't learn a new framework and create such an awesome app with it in just two weeks. I'll rewrite it in J! sometime over the next couple of months, time permitting. (I also had to use jQuery, but I'll switch it to Mootools in a couple of days)

So that I submit something before the competition deadline, I've uploaded just about everything to http://bitmeta.org/ghop-joomla-031.zip, including the JTML stuff (my actual unit test plan, some other sample XML files, WikiML of the basic planning I did for it etc.) and the JUserTest stuff (PHP/HTML/CSS/JS).

Regarding further development, Marieke had quite a few excellent suggestions on how the system could be improved, so I'm working on implementing those at the moment. I may just have the next version ready by tonight :)

As this is an open source, community project, and an ongoing project, I figured it would be a good idea to use this project to get more people involved with Joomla!, outside of GHOP. Seeing as J! is free software, and Wikiversity is free content, they seem like a good match. I've always been looking for a project with which to run a Wikiversity PHP/MySQL in FOSS, and JUserTest looks pretty good. I've started some basic planning at http://en.wikiversity.org/wiki/JUserTest and I'll start preparing documentation in time.

Anyway, back to the code. I advise you don't try installing it if you haven't used CodeIgniter before; I'll submit installation instructions in a moment. The system is running on my server and I'm happy to demo it to anyone with MSN/GTalk/Skype who would like to see it. However, for now, here are some screenshots to demonstrate the system a bit better.

Image

Image

Image
(Importing does actually work, but the "import succesful" page is really boring, so I figured I'd show you this instead.)

Image

Image

Image

And that's pretty much it. More fancy UI stuff coming soon.

Anyway, please reply with questions/comments/praise/abuse/rum.

-Draicone

P.S. Amy: I couldn't seem to reach you over Skype, I think I'm having network problems with it. I'll try again in a couple of days.

User avatar
mcsmom
Joomla! Exemplar
Joomla! Exemplar
Posts: 7897
Joined: Thu Aug 18, 2005 8:43 pm
Location: New York
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by mcsmom » Sun Feb 03, 2008 2:38 am

Looks nice Draicone, very nifty.

Of course, long term making it a 1.5 component with mootools is what you should be doing. :)
But this is an extremely nice basis for that.
So we must fix our vision not merely on the negative expulsion of war, but upon the positive affirmation of peace. MLK 1964.
http://officialjoomlabook.com Get it at http://www.joomla.org/joomla-press-official-books.html Buy a book, support Joomla!.

AmyStephen
Joomla! Champion
Joomla! Champion
Posts: 7018
Joined: Wed Nov 22, 2006 3:35 pm
Location: Nebraska
Contact:

Re: 031 - Create a unit test plan for the Module Manager

Post by AmyStephen » Mon Feb 04, 2008 4:15 am

Draicone -

I imagine your mind is always creating. I'll bet when you (finally) lie down to sleep at night and even as you sleep, your brain is still processing ideas and possibilities that refuse to release you until you deploy them.

Keep exercising your creativity. You will always be happier and you will always be more successful in roles where you are given lots of freedom - seek those out - find people who respect that in you and see it as the gift it is. AND, as importantly, find others to help balance that out and listen to them when they talk to you about deadlines and scope creep and deliverables and "that is not what they are paying us to do!" - but, make certain they respect your obvious creativity and that you are not asked to deny it. It's not always going to be easy because most people will want you to conform - and you don't do that well. Remember - since most people do tend to conform - we got that covered - you should be who you were born to be - that's what we need.

I warmly, and with more than just a tiny bit of a chuckle, receive your Unit Testing for the Module Manager task, that yes, includes a well written Unit Test for the Module Manager, but only because basic test data is essential for a larger task you defined yourself  for a Unit Testing Management and Deployment system written in a MVC Application Architecture, yes, just not the one that is produced by the open source project whose task it is you are doing, but pretty amazing, none the less.

I look forward to seeing what you accomplish as you go forward from here.

Kind regards - task closed - thanks for participating with Joomla!,
Amy :)
Last edited by AmyStephen on Mon Feb 04, 2008 4:17 am, edited 1 time in total.

Draicone
Joomla! Apprentice
Joomla! Apprentice
Posts: 20
Joined: Sat Dec 01, 2007 7:33 am

Re: 031 - Create a unit test plan for the Module Manager

Post by Draicone » Sun Feb 10, 2008 12:40 pm

Thanks Amy!

I look forward to working with Joomla! in future. Now, to start submitting patches and join the dev wg... :)


Locked

Return to “Google's Highly Open Participation Contest”