Tag Archives: wordcampdallas2008

WordPress Testing Presentation Video

Posted by Jacob Santos on April 13, 2008 2 comments

Critique of Myself

Crap.

I think I found some things out watching the video:

  1. I should take Johnathon Bailey’s advice and do like he does when giving a presentation. He’s rocked, mine was barely passable.
  2. I need to lose weight, so that I don’t have to pause to breath every other two words or so.
  3. You don’t need to look at the projected screen, when what is on the screen is right in front of you. I kicked myself as I was doing it, “Why are you looking at the screen self? It is right in front of you!” However, it probably isn’t something I can fix.

It is funny, I don’t remember taking as many pauses, I remember a few because I had to double check my notes on where I was at and what I needed to cover, but damn, the monotony makes me bored just listening to myself. There were a couple of times when I said a sentence without pausing, but probably only once or twice.

I had a similar problem the first couple of times I was on the WordPress Weekly Podcast. I couldn’t get my words out, but at least I didn’t stutter and say, “Um” more than probably three or four times. I find that I’m so used to making hand movements, when I speak that I can’t talk without them. I’m conducting myself speak.

Answering John P’s Question

I also have problems with either… or questions, as John P found out. I can hear my co worker now:

“Exactly.” I replied.
“It was an either… or question, so you have to choose one or the other.” said my mind bringing up a memory from several months ago.

To answer John’s question:

The Automattic WordPress Tests will allow a user to download, run the tests, and see which parts of WordPress is going to break. This, I envision, will shorten support time, decrease the scope of support questions, and improve the user outlook of WordPress. Instead of asking many questions about the host, the server configurations, PHP configurations, and asking the user to try many different techniques, the first support answer will be to download the WordPress Automated Tests and run it to see what breaks in WordPress and report back.

The amount of failures will be different on most users servers, so perhaps Alex and the other contributors don’t see any failures, which would be a good thing. Sending them the failure report or going to that web site which runs my tool will be able to drill down solutions and patches for fixing the problem the user is having.

This assumes that there is at least 80% code coverage with the test suites of the WordPress Library. RIght now, it would be difficult to say, “Just download this and run it,” until that point in the future when above 80% code coverage has been achieved.

So yes, as a user, you will be able to in the future, download the Automattic WordPress Tests and run it to see what breaks and find out how well your host supports WordPress. The Automattic WordPress Tests also has to be easier to install and run.

Why patch_cron.php?

What happens is that the wp-cron.php checks for wp-config.php on the base folder. So this means that there needs to be a wp-config.php in the folder that you are executing WordPress. This is the reason why the Automattic WordPress Tests requires that you create the wp-config.php in the folder that run-tests.php is in.

My Presentation

I do want to thank the hard work of the engineer that spent the time recording the videos over the weekend. Also, John P for posting my video.

Really, I wanted Alex Shiels to give the presentation and I hope he does at some point. My presentation was just an overview of User and Automated testing, so I hope that people either find the information they need themselves if they are interested or another presentation which drills down what is needed.

Possibly Related Posts:


Talk at WordCamp Dallas 2008

Posted by Jacob Santos on April 3, 2008 Comments Off

I think my session has more in common with Mike Naberezny’s talk at OSCON. The video will be coming out, but you’ll probably realize that Mike knows more than me on the subject. I am just guessing, but you can compare and contrast.

My focus was to get more people to build test cases for the Automattic WordPress Tests automated test suite, which focuses more on integration testing (what I commonly call functional testing, albeit inaccurately, but it is a habit).

I look forward to slides for Mike’s talk, which hopefully is released afterwards. I’ll like to be able to go next year, if another testing methodology presentation is given. I’ve been learning about automated testing for the past year and what the different methods can offer. I hope to graduate to understand what smoke tests and all of the other testing that Mozilla does. It would be interesting to apply those methods to other projects of mine and find out whether or not the quality improves.

My hope is that such a talk won’t be needed again next year, because WordPress would have finally grown up and completed several Quality Assurance projects. It would be nice to not have to read how poorly written the application I use is, because then I feel as if they are somehow implying that my code is also bad.

As an aside, when did Planet PHP syndicate, so many great blogs? Wow, just wow.

Possibly Related Posts:


Testing With WordPress Presentation

Posted by Jacob Santos on March 31, 2008 5 comments

My time was pretty much cut short before I was able to fully talk about all that I wanted to talk about, so I’ll apply more clarity to certain parts that may need it. To be honest, I didn’t think I would have enough material to cover the full hour, but from how it went it appears that I could have gone on for two hours and still not completely covered the full scope. I could have also done without the user testing and focused completely on the automated testing portions, since the majority of the people who weren’t developers left anyway.

I was probably better off having a smaller audience, because I wasn’t as nervous. I think I did well, once I talked past the introduction and became comfortable with speaking to the audience. I’m going to have to watch the video to make sure that is the case and I will update this post if there are any other parts I need to explain better.

The WPTests Tool

I think the first part is what my tool, WPTests is and what it does, and what it is meant to do. The WPTests tool is to check the status of the Automattic WordPress Tests without having to download Subversion, set up Subversion, set up the config, and fix other issues. I don’t include a readme or help, and I only fix one issue that I have.

My intention was to send Alex my patches for the Automattic WordPress Tests and then my issues would be fixed and so would others that use my system or just download that system. I also wanted a tool which would work on Windows and allow me to record the progress. I do need to optimize the SQL queries in order to increase the performance or at least create pages based on dates. This would allow for getting all of the test runs in a single day to be displayed and then have the last test run for that day for all of the other days.

The tool provides a history, to give context to what the failures mean. If all of a sudden the tests all fail, then you know you shouldn’t upgrade. The problem is that the test runs are only once a hour, two minutes past the hour, so an issue might be fixed within that time, but the person would have to wait an hour or slightly over an hour before they would be updated.

It would also be great to have historical data on what tests failed and when and when the test passed. I like the tool, but I have plans for another, which are more personalized for the user, which will allow for choosing which test to run, which plugins to run, different options that the Automattic WordPress Tests supports and finally see the results of that test run.

Both tools are based on the same code, so it isn’t difficult to maintain both and I do plan on maintaining both. The tool I presented at the session was mainly to prove to Nikolay of Automattic of my intentions and what I wanted, which is something akin to continuous integration, but not exactly.

The tool can be download at wordpress.svn.dragonu.net/WPTests and you will need Subversion to do so. I do not have tags, so you just check out the root and you are ready to go. I do still plan on giving this to Alex and hope that he accepts at least part of it in to the Automattic WordPress Tests.

My WPTests tool does not support PHP4, you must have PHP5.1+ and PDO_SQLITE extension enabled to run the tool. You can however change the DSN and since all of the SQL is supported by MySQL also, you shouldn’t have issues. What you will have issues with, if you switch over to MySQL is that I use the TEXT field type, so a lot of space will be taken up. I’ll change that later, so that both are supported at a later date.

Amount of Test Cases for a Function or Method

The one failed test case was not the only test case covering that method. If I did not have that test case, then I would not have been able to figure out that my method was not working correctly. For that method I had four test cases covering about 3 lines of code (I’m being serious). For the entire class, I had a total of 15 test cases covering 4 or 5 very small methods.

When you write test cases, you want to write at least one test for valid input and at least one test case for invalid input. For each branch (if statement, while loop, for loop, anything that might require brackets) needs to have at least one test devoted to just that.

So all of my test cases covered almost all of the possible cases that a user will be using my methods for. I also thought ahead and thought of twists to make sure that my assumptions weren’t wrong and then I’ll have issues down the road. The failed test case was an assumption I had that I was unsure about. I was providing support for PHP4 and I can’t exactly test with PHP4, because of the way I have the PHPUnit library set up and what extensions I require. In theory, it should have worked perfectly, however that was not the case.

It is perfectly okay to write just one test case per function, but do realize that with one test case for one function, it might give false impressions over the reliability of that function. A function having one test case is only useful if that function only has one line and is calling another function and that is all it does.

You also want to split assertions made in a test case into multiple test cases, if you have more than one assertion in a test case. The reason is that, once there is a failure of an assertion in a test case, no other assertions are tried in that test case. If you have 5 assertions in a single test case and the test case fails after the first one, then the other 4 will not be reported as failures or even show up in the total. If it makes sense, move the other 4 assertions to their own test cases. If the assertions are dependent on each other, then by all means include them in the same test case.

Skipped and Incomplete Tests

I had to skip over the explanation of both the skipped and incomplete tests and how to do that. There is an issue I have with the current Automattic WordPress Tests automated test suite, in which test cases exist for a nonexistent function. The solution to skip over that test is to set a constant to false. That is not the correct way to solve that problem, because I want to run the other known bugs and you should always run the known bugs no matter what.

The correct way it should have been done is to test for whether that function exists and if it doesn’t, then mark that test case as skipped.

Skipped tests aren’t so much a problem as failed tests and errors, because it just means that whatever the test cases was using is not supported and it can’t run without it. If that component is important, then it probably should bring down the entire test suite.

Incomplete tests are test cases, which the developer started, but did not yet finish. So you can use this to your advantage, you can design all of the test cases you want for a component, mark them as incomplete and then go through and finish them all later. I would just as rather not do that, because you shouldn’t have a lot of incomplete tests and mostly it doesn’t take long to look through a function and write test cases for that function or method.

What the Number of Failed Test Cases Means

I don’t think the explanation I gave matched what I wanted to say. If there is a failed test case, then you should assume that there is a problem. That is until you look at both the test case and the function or method it is testing. If you find that the problem is the test case, then the test case should be fixed. If you find that it is the function or method that is the problem, then it is a bug and needs to be filed. However, both the test case and what it tests should be looked at.

What I said, was basically, the formatting test suite covers a full range of test cases. Not all of those test cases are perfect. Some are feature requests, which haven’t been filed and probably won’t be fixed. Therefore, you can safely ignore those. There are also other problems, where the actual test case is wrong and needs to be fixed. There are or were other test cases which actually found bugs in formatting functions, but it has been a while, so I don’t know if they have actually be fixed or looked at.

It can also be assumed that unless the entire automated test suite is brought down in a fatal error, that you can successfully upgrade WordPress. It does mean that there will be issues with bugs or regressions, but those should be bought to the developers attention.

Names of the Methods Should Mean Something

I’m in the camp that method names should mean something for that test. Meaning I should be able to read the test case method name when it fails and get an overview of what happened before I actually look at the test case and function to see where the problem is. It is a convention and with any convention it is up to the developer. If you want to write the function name and just have test_function_name2() and test_function_nameN(), then by all means do so.

What is important is not the method name of the test case, but that the test case exists. I will not fault you for naming it how Alex does it in the Automattic WordPress Tests. Alex might even change my way of naming the test case method names to something he likes. I am not an elitist on the subject, even though I did slightly mock Alex during the Session, even if I didn’t purposely mean to. My purpose was only to question, which way you would do it and finally advocate mine.

You can name it any way which the test case name makes sense. Either way, I’ll be happy if you just do it.

A List of “Thank Yous”

  1. Charles Stricklin – I wanted to thank him for allowing me to speak at the Fisco, Dallas WordCamp 2008, even though pretty much no one there knew who I was and for giving a really geeky presentation.
  2. I want to thank Branson Tourism Center and Branson.com for sponsoring my trip down there. For my work to do that, is awesome. I don’t think some of the other places I’ve worked out at would had extended their hand (and wallets) in that way. Larry and Lianne are really great people to work for and I would say that, even if they fired me tomorrow and say that years after I do finally leave. Of all of the people I’ve worked for, there were very few who I actually appreciated working for and would do anything for. It isn’t just that they are nice, they are, but it is the respect and that is the most important when I work for a company.
  3. I would like to thank the city of Frisco, Texas for allowing WordCamp Dallas 2008 to be held there and appreciate the technician who ran the cameras to record all of the sessions. That was really great of the city to allow all of us to use their facility for the event. The city is just really beautiful.
  4. Everyone else who stayed for my session, thank you. I also thank all of the people who I talked to and told me about theirselves.

To Be Continued…

This post will be updated at a later time, when the video is available and I can go over more of what I missed and didn’t fully explain.

Possibly Related Posts:


WordCamp Dallas 2008 Friday Night Mixer

Posted by Jacob Santos on March 30, 2008 4 comments

I posted some pictures to the Gallery.SantosJ.Name, but I figured after watching Matt show it off, that I would try it for myself. I have to say, it actually got passed my loathing of using HTTP uploading. This is the way it should be done!E

The gallery were for people of the WordCamp Dallas 2008, who were in Friday night and wanted to hang out before the big event. I have to say that I enjoyed talking to everyone that I had the pleasure to talking to. It isn’t just about Matt! It is about Lorelle! Okay, maybe not completely, but she was there and very active. I wish I moved around that much, I probably wouldn’t be packing as much weight.

Possibly Related Posts: