Jesse's Blog

Another Bug Fix Part 2

2009-04-17T14:42:00.003-04:00

This part describes my process for fixing the bug.

As I mentioned in part 1 I believed that the problem simply came down to refreshing the UI after the parts list was updated. So the questions I needed to answer were: where could I find the UI refresh method and where could I call it that would make the most sense?

Answering the first question was simple enough. After some light searching I came across the ExtensionDetailsViewer.java class. This class was responsible for generating the UI for the Extension Details section. In this class was a refresh() method. Perfect.

As for the answer to the second question, this was also easy enough. I simply needed to find out where the "Specify Parts" dialog was being initiated and include a call to refresh() after that.

After looking at the code for awhile trying to understand it, I came across a few lines that interested me located within a method called createButtonControl().


Button button = new Button(composite, SWT.NONE);
GridData gridData = new GridData();
gridData.heightHint = 17;
button.setLayoutData(gridData);
button.addSelectionListener(internalControlListener);
button.setData(ITEM_DATA, item);
button.setData(EDITOR_CONFIGURATION_DATA, configuration);

I thought this might be the button I was looking for (the one that opens up the Specify Parts dialog). I tested it by setting the button's text to something different and sure enough it changed, so I knew that this was the right button.

I took note of the addSelectionListener() call, as the listener that was passed was most likely what opens the dialog. The internalControlListener was an instance of the InternalControlListener class, the contents of which were actually defined inside ExtensionDetailsViewer.java. Looking at the class, I came across a method called widgetSelected()


public void widgetSelected(SelectionEvent e)
{
   // for button controls we handle selection events
   //
   Object item = e.widget.getData(EDITOR_CONFIGURATION_DATA);
   if (item == null)
      item = e.widget.getData(ITEM_DATA);
   if (item instanceof DialogNodeEditorConfiguration)
   {
      DialogNodeEditorConfiguration dialogNodeEditorConfiguration = (DialogNodeEditorConfiguration)item;
      dialogNodeEditorConfiguration.invokeDialog();
   }
   else if (item instanceof ExtensionItem)
   {
      applyEdit((ExtensionItem)item, e.widget);
   }
}

I took notice of the if statement that checks if the item is an instanceof DialogNodeEditorConfiguration. I suspected that it was here that the Specify Parts dialog was called. So I put a breakpoint on the invokeDialog() call and traced it. Sure enough, when I clicked on the "..." button I was stopped at this call. I stepped through it and ended up at a class called SOAPSelectPartsDialog.java which was responsible for creating the Specify Parts UI. So I knew then for sure that this line is what started the whole process. To test my original theory I inserted a call to refresh() right below the invokeDialog() call and ran the application. As I suspected, the UI was now fixed! After updating the parts and clicking OK in the Specify Parts dialog, the text in the parts field changed without me having to change focus.

You can find my patch on Bugzilla.

Another Bug Fix Part 1

2009-04-17T14:07:00.009-04:00

I had mentioned earlier that this bug seemed a little more complicated. As it turns out, the fix was equally as simple as the previous bug.

Since I haven't given much detail on this bug yet I'll divide this post into 2 parts. The first part will be about re-creating the bug, the second part will describe how I fixed it.

As you may recall from my previous posts, the last bug I worked on dealt with the extensions tab of the WSDL editor. This bug deals with the same area of the UI, however unlike the previous bug this one is related to the Extension Details section rather than the Extensions tree list.

The problem in this section related to one attribute known as "parts." Parts seems to be a list of objects that makes up the soap:body extension. As you can see in the image below, the soap:body of the example WSDL I created has 2 parts, "parameters" and "NewPart"

At first nothing seems wrong with this scenario. However, to find the bug we have to click on the Button next to the parts field labeled "..." which pops up the "Specify Parts" dialog.

Next, I unchecked NewPart and clicked OK. Now we can see the problem. In the image below, notice that the parts field still displays "parameters NewPart". The NewPart is still included as a part according to the UI.

Now, as you can see the focus has been lost from the soap:body extension. I re-clicked it, and the results are below.

Now the parts field simply displays "parameters". The UI is now behaving correctly. The fact that there wasn't any actual problem updating the parts list told me that the problem was simply the UI not updating after the parts list was updated. The fact that the UI fixed itself after I changed focus back to soap:body told me that the UI perhaps just needed to be refreshed. I'll describe my fix in Part 2.

Update, New Bug

2009-04-10T18:47:00.002-04:00

It's been awhile since my last update. Unfortunately some other courses of mine have taken up much of my time and I've had to put Eclipse on the backburner.

My previous bug still hasn't been reviewed it seems, so I'm still waiting on that and I hope I get some feedback soon.

In the meantime, I figured it didn't make alot of sense to end my semester with one bug completed a few weeks early, so I decided to take a look at Bug 235381 to see if there was any work I could do on that.

Although I didn't find a lot of time to look at it in-depth, I quickly noticed that the code for the "Parts" attribute seems quite a bit more complicated than the Delete button from the previous bug. The code to populate, modify and display this list spans across a bunch of files and it was hard to determine where exactly the problem was. While at first glance it simply seems like a UI refresh problem, it might actually be more complicated than that. I will go into further detail early next week and hopefully make some sort of progress on this.

Problem Solved

2009-03-25T20:51:00.004-04:00

In my last post I talked about my initial fix for Bug 209289 which I thought was a cheap fix since it didn't get to the real root of the problem which was solving why the refresh() method needed to run through every time in order for the delete button to be disabled. Why didn't the state of the button persist even if the refresh() method wasn't called? Clearly there was another piece of code modifying the button's state.

I tried debugging for awhile, running multiple stack traces to try to find the point where the button's state was being changed, with no luck. Along the way though, I came to understand the code more and more.

It hit me that AbstractExtensionsSection.java was a subclass of AbstractSection.java, so I decided to try finding something in that file. Unfortunately I found nothing, but I realized that the name "AbstractExtensionsSection" implies that there is probably a class that subclasses it. I ran a search and I came back upon W11ExtensionsSection.java, the file I originally referenced in my January 29th post and subsequently dismissed on my February 24th post because it only called the setEnabled() method on addButton and removeButton. Originally I figured that this couldn't have been where the problem was, that it had to be wherever the button was created and it's state was set. However now I had a better understanding of the code.

Looking at the method that the code was inside:


public void setInput(IWorkbenchPart part, ISelection selection)
  {   
    super.setInput(part, selection);
    isReadOnly = true;   
    ExtensibleElement extensibleElement = getExtensibleElement(input);
    if (extensibleElement != null)
    {   
      Element element = extensibleElement.getElement();
      if (element instanceof ElementImpl)
      { 
        isReadOnly = false;
        modelAdapter = WSDLModelAdapter.lookupOrCreateModelAdapter(element.getOwnerDocument());
        modelAdapter.getModelReconcileAdapter().addListener(internalNodeAdapter);
      }        
    }      
    addButton.setEnabled(!isReadOnly);
    removeButton.setEnabled(!isReadOnly);        
  }

I realized that enabling the two buttons here didn't make any sense. The logic for this is handled in the refresh() method in AbstractExtensionsSection, and if the state changes refresh() will run through again and adjust the state accordingly. So why does this method need to handle the buttons' state? It doesn't.

So, I removed these two lines from the method and ran the application. To my delight, the changes worked, and the results I saw using my cheap fix a couple of weeks ago were the same changes I was seeing now. This time, I did it without having to change refresh() execute every time the extensions section was returned to, which is a waste of resources. Instead my change simply leaves the handling of the buttons' state to the refresh() method, which I feel alot more comfortable with.

So now I need to take this solution to Bugzilla. Hopefully I'll be able to get my patch into the tree. Also, my next step is to find another bug to work on, as I have no intention of sitting around for the final few weeks of the semester.

0.2 Release

2009-03-11T18:11:00.006-04:00

Well, I believe I'm ready to release 0.2. If you read my last blog, I was currently evaluating what changes I need to make between two files (AddExtensionsComponentDialog.java, AbstractExtensionsSection.java). I figured out that AbstractExtensionsSection.java was the file I was looking for.

I ran a find for "removeButton.setEnabled" (the method which enables and disables buttons) within this file and I found two lines, one of which was located within a method called refresh(). The name alone suggested that this might be relevant. It would make sense for the section to be refreshed on a consistent basis. Here is the code:


public void refresh()
{
     setListenerEnabled(false);
     if (input != null)
     {
          if ( prevInput == input)
               return;
          else
               prevInput = input;

          Tree tree = extensionTreeViewer.getTree();
          extensionDetailsViewer.setInput(null);
          tree.removeAll();

          addButton.setEnabled(!isReadOnly);

          extensionTreeViewer.setInput(input);
          if (tree.getSelectionCount() == 0 && tree.getItemCount() > 0)
          {
               TreeItem treeItem = tree.getItem(0);
               extensionTreeViewer.setSelection(new StructuredSelection(treeItem.getData()));
          }

          removeButton.setEnabled(tree.getSelectionCount() > 0 && !isReadOnly);

          // Bugzilla 197315. Make this bulletproof for maintenance release.
          Control detailsViewerControl = extensionDetailsViewer.getControl();

          if (detailsViewerControl != null && !detailsViewerControl.isDisposed())
          {
               detailsViewerControl.setEnabled(!isReadOnly);
          }
     }
     setListenerEnabled(true);
}

The instance of Tree represents the list of extensions, which is highlighted in the photo below:

And as I mentioned before, removeButton refers to the Delete button which I am trying to fix. At first I believed that perhaps refresh() simply wasn't getting called when switching tabs e.g. clicking on Extensions, then clicking Documentation, then clicking Extensions again.

So I inserted a breakpoint at the beginning of the method and ran a stack trace. Turns out refresh() actually DOES get called whenever the tabs are switched. So why is the button not being re-disabled? This isn't even the real issue, which I'll explain later.

I then realized that even though refresh() was being called after every switch, the method was being exited early. If we look at this code:


if ( prevInput == input)
     return;
else
     prevInput = input;

Notice here that if "prevInput" is not equal to "input" then the method should return out. I realized that prevInput refers to the previous status of the Extensions frame. It was being compared to the current status of the frame, and because switching tabs wasn't changing the actual contents of the frame, the method was exiting early. To change this, I changed the if statement to look like this:


if ( prevInput != input)
     prevInput = input;

When I ran the Eclipse Application, the button behaved properly!

Note that I switched tabs before switching back to the Extensions tab in this picture. When I clicked on an object in the WSDL editor the buttons kept their proper state. It would appear that the problem is solved.

However, to me the real issue is: Why does refresh() need to be called every time to persist the state of the buttons? Why don't the buttons remember their state? I believe that the answer isn't that the buttons are forgetting their state, I believe it's because something else is changing their state. This would suggest that the problem actually doesn't occur in refresh(). The change I made now is a bit of a cheap fix, what I would like to do is eventually find the root of this problem and fix it, which is why I don't plan on submitting a patch yet.

Clearly the developers wanted to make sure that the code executed in refresh() wasn't done every time a tab was switched. Thus, the problem with my solution is that by cutting the old if statement out, I ensure that the code inside refresh() is executed on every tab switch, which is a waste of resources. So for 0.3 my current objective is to find the underlying problem as to why the Delete button's state change isn't being persisted, and that means finding out where else the button is being changed.

Its been awhile! Update

2009-02-24T11:53:00.003-05:00

Well, first I'll briefly explain my absense. I had originally been under the assumption that I was following the DPS911 release schedule (which is once every 2 weeks), turns out after talking to Dave i was actually following the DPS909 release schedule which was a huge relief. Because of my earlier assumption I had already done alot of work on Eclipse and had put myself ahead of the rest of the class, so I decided to leave it for a couple of weeks and focus on other courses.

Now that the rest of the class is around where I am, I feel it's time to get back to work on this thing. Currently the objective is to find source code thats relevant to our bug. If you've read my post from a few weeks back you'll know that I thought that I had found some relevant code but I couldn't verify this because I had problems getting the WSDL editor to load. I found out that it was because the code I was checking out in the HEAD tree inside the CVS repo wasn't matched up with the version of WTP I was running, so I checked out code from an older tree, problem solved.

Next I needed to test whether the "removeButton" Button was the same as the Button I was looking to change. I added a line of code to set the removeButton's text to "Test".

removeButton.setText("Test");

Next I ran my Eclipse Application. Looking at the WSDL editor, my button had it's text changed to Test. Success!

However, there was a slight problem. This removeButton was referenced only one time in the W11ExtensionsSection.java file that I was working in. This basically told me that this wasn't the file I wanted to make my changes in.

Fortunately for me, I able to attend a presentation by IBM's Angel Vera last week. During the presentation, Mr. Vera showed us some techniques that we could use to find code using the "Search" feature of Eclipse. Using this, I searched for "removeButton" within the org.eclipse.wst.wsdl and org.eclipse.wst.wsdl.ui packages. To my suprise, the only reference to removeButton was the one that I had already identified, meaning that this Button was declared in a different package. Fortunately, this package was easy to identify. I simply had to mouse over the removeButton reference inside the code and it led me to the org.eclipse.wst.xsd.ui package. I checked this package out and ran a search on it, and found many references to the removeButton, mainly in 2 files (AddExtensionsComponentDialog.java, AbstractExtensionsSection.java). I now realize that I am probably going to have to make changes within one of these 2 files or maybe even both in order to fix my bug.

So, that's currently where I stand. I know where I need to change things, but I don't know yet what changes I need to make. I feel confident that I'll be able to figure things out.

Some Success, Some Roadblocks

2009-01-29T18:17:00.002-05:00

Well firstly I believe I've located some relevant code. The page for the bug I'm working on shows that the component is wst.wsdl. Using this information I happened upon 2 packages: org.eclipse.wst.wsdl and org.eclipse.wst.wsdl.ui. I hoped that these packages would contain some code that could point me in the right direction, and fortunately I found one file that could prove interesting. Under the UI package I found another package called org.eclipse.wst.wsdl.ui.internal.properties.sections. Within this package I found a java file called W11ExtensionsSection. Success? I hope so. I feel like this could be a big break since within this file I found references to two buttons called addButton and removeButton. As my previous post explained, the section of the UI I'm focusing on has Add and Delete buttons, so these references could possibly have something to do with those 2 particular buttons.

I've been unable to check as to whether or not the code is relevant. In my previous post I showed a screenshot of the WSDL editor in my main Eclipse window. When I try to debug in a new Eclipse Application, for some reason the WSDL editor isn't there. I've decided to look to the Eclipse Webtools Newsgroup for help, so hopefully in time I'll have the answer to my problem.

I've thought about what I can submit for a 0.4 Release. I definitely won't be able to make a patch for this fix until a later release, so I'm not sure what I can create in-between. Perhaps a simple patch making a slight change to the UI? I'll have to discuss this with Dave.

Switching Bugs

2009-01-27T17:08:00.004-05:00

Well, after some difficulty (mostly due to the slightly vague instructions) trying to re-create Bug 142301, I managed to get it done. However, it seems like the bug has already been fixed.

It's hard to tell since the bug is from 2006 and I imagine the UI has changed since then, so some of the page names in the wizard (ie. "Web Service Java Bean Identity Page") seem out of date. I originally stumbled on this page which is a guide on how to use the Web Service wizard, from the screenshots it didn't seem like there was a "Web Service Java Bean Identity" page, thus I figured I wasn't in the right wizard.

I tried the WSDL wizard since the bug page mentions attempting to improperly name a .WSDL file. In this wizard I found what I was looking for: a text box for naming the WSDL file. However, I found that my bug had already been addressed. As the top of the image shows, the necessary error message and disabled buttons appear when I try to add an * to the filename.

It's frustrating that I wasted my time on this bug, but I decided to trek on and luckily I stumbled upon another bug that caught my fancy. This description was even less informative then the last one, but looking at the Component and noticing that it was in wst.wsdl meant that it had something to do with a WSDL file.

I had already created an EJB Session bean when trying to re-create the first bug with this tutorial, and using that I created a Web Service with a WSDL file. I opened this file, checked the extensions tab in the properties window and was distraught to find that this bug also seemed to be addressed as the Delete button was disabled when according to the bug it shouldn't have been. However, when I switched to another tab and then back to the extensions tab I found that the button was in fact enabled when no extensions were present. Somehow changing focus between tabs re-enabled the button.

Well, I'm glad I at least got the re-creation of the bug out of the way. Now, my next problem is to find out where in the code this is occurring. I feel like if I can do that the actual change in code wouldn't be anything crazy, but then again with a big, unknown system like Eclipse you never know.

0.4 Goals

2009-01-24T12:36:00.002-05:00

For my 0.4 Release, I've been trying to find a bug to work on that could provide me with a good introduction to the Eclipse system. I came across Bug 142301, which states that there is a fix required for one of the wizards. The fix must not allow the user to enter an invalid filename (ie. something with *&^%$ characters). If the user attempts to enter an invalid filename, the Next and Finish buttons are to be disabled and a message needs to appear to notify the user.

I feel like this is a good bug to start off with, and thus I will be working on it for my 0.4 (and perhaps in future releases as well depending on how successful I am).

One thing I find disheartening is that there seems to be no IRC channel for Eclipse developers. I'm worried that I won't be able to receive the same kind of support from experienced developers that I enjoyed with Mozilla. However, Jordan has pointed me towards the Eclipse newsgroups, which apparently have plenty information. I hope so because I have no clue how to re-create this bug or where I can find the relevant code.

Where do I Begin?

2009-01-24T08:39:00.002-05:00

I don't know where. I guess I'll start with my last semester results. My patch for the source server project was accepted into Mozilla's tree, which I'm very proud of.

My results with the source server project make it all the more confusing as to why I'd be so hesistant to work on my new project. I won't spend this entire post psychoanalyzing myself, but sometimes I think it goes beyond simply being lazy. I wonder if it has something to do with my self-confidence. This was a problem last semester with the source server project. It took me so long to finally take a look at the system and get started, and this semester with my new project it's the same.

Oh yes, my new project. I will be working with Eclipse WTP this semester. Dave recommended it since I was looking to try something a little different, and Eclipse is all Java.

Like I said, this happened last semester. At the time I believed it was just me being lazy, but now that it's happening again I'm starting to wonder if it has something to do with my confidence as a programmer. It's wierd that I would feel that way still, I mean I felt the exact same way last semester but I overcame it and once I got into the project it was easy for me to keep going. But now here I am and the cycle continues. Like I said I attributed it to laziness originally, but now I believe that perhaps by understanding why I do it a little more, I can overcome it.

I don't want to make baseless excuses for myself but I seriously believe that in this case it might be true. I've never really thought of that before but now that I'm aware, I hope that I can fix it starting with this project, because I know I'm a capable programmer and I know that I have the ability to learn a new system and contribute to it.

I will be updating my blog with my 0.4 release info shortly, I figure it's a better idea to keep this stuff a seperate post.

I'm still not too warm to this whole blogging thing, however it's actually felt nice to get that off my chest.

0.3 Release Submitted

2008-12-17T15:02:00.004-05:00

Well, I've submitted my third patch, which I will be considering as my 0.3 release. Whether or not it gets accepted remains to be seen but I feel like I've done enough to submit a good 0.3.

Most of the issues I needed to correct were minor things such as naming conventions, however there were also some other issues that ted pointed out that I needed to fix.

The first was a change to the data block. Ted didn't like the fact that the SRCSRVTRG variable was set to HTTP_EXTRACT_TARGET. Instead he wanted this variable to point to a local target because SRCSRVTRG was supposed to resolve to the path that the file is extracted to. What I found is that while this is true for a CVS extraction, I believe it serves a different purpose when used for HTTP extraction.

When I run srctool on a pdb using the old CVS implementation, I see entries like this:

[e:\fx19rel\winnt_5.2_depend\mozilla\layout\style\nsrulenode.h] cmd: cvs.exe -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot checkout -r 1.64 -d 1.64 -N mozilla/layout/style/nsRuleNode.h

As you can see, the header file has a "cmd" which is run to retrieve the source file. The SRCSRVTRG is then set as a local path to tell srcsrv where to dump the file. However, in my HTTP implementation srctool showed this:

[c:\mozilla\src\toolkit\xre\nsupdatedriver.h] trg: http://hg.mozilla.org/mozilla-central/raw-file/3611062ed098/toolkit/xre/nsUpateDriver.h

As you can see, the files no longer use a "cmd" but a "trg". I wasn't sure if this was related to HTTP_EXTRACT_TARGET or SRCSRVTRG so I experimented by changing SRCSRVTRG to ted's recommendation, and srctool showed this:

[c:\mozilla\src\toolkit\xre\nsupdatedriver.h] trg: C:\mozilla\symbols\xul.pdb\....

The trg was now the location of the xul.pdb file. This showed me that SRCSRVTRG was being used to determine where the file was located, and thus was being used differently than if the source server was using CVS. Testing in Visual Studio proved this, as the source server didn't know where to find the file. So, to sum things up, SRCSRVTRG had to remain the way it was.

I was also asked to remove the SRCSRVCMD command from the data block, as it wasn't being set to anything. I assumed I needed it as the srcsrv documentation said it was a required variable, but it turns out that I didn't need it so I was happy to remove it.

So, I was ready to submit a new patch when I found out that my entire solution didn't work in Visual Studio 2005! I had been testing on 2008 the entire time and it worked like a dream, however 2005 didn't seem to know how to do HTTP retrieval. After discussing this with ted and sending him my stuff, he got it working fine in WinDBG (I had tried WinDBG myself without any success but I must have done something wrong in the setup) so in the end not working in 2005 was ok with ted, he speculated that perhaps srcsrv wasn't properly implemented in 2005 and I'm inclined to agree based on what I've seen in my testing.

So, now that I've resolved these issues I feel confident about getting my project implemented in the tree. Maybe not with this patch, but eventually.

0.3 Update #2, failed review again :(

2008-12-09T14:51:00.002-05:00

Well, it seems ted has found some more problems with my patch so I'll have to try again. I'm still very optimistic that I can get this done on time however I'm in the middle of exam week so it will take some time before my next patch is created. Stay tuned.

0.3 Update

2008-12-01T18:42:00.003-05:00

Well, it's been awhile since my last post but I'm in overdrive mode for some of my other classes so unfortunately I've gotten a little behind on my 0.3. Luckily, I believe I'm close to having my patch accepted.

I guess I should explain that my first review didn't go so well. Ted pointed out a few things that he didn't like about my patch (one of which was a dumb mistake on my part) but fortunately none of the required changes was difficult (mostly naming conventions and a syntax error inside of the data block) and Ted even mentioned that my patch was close. So, I've uploaded my next patch and I'm hoping for the best.

0.2 - Finished

2008-11-14T19:01:00.005-05:00

Well, after a lot of trial and error I got my local server to serve the pdb files properly.

The big problem I was having was that although the symbols were being loaded, when I tried to retrieve source code it would only ask me to locate my local source files and not try to retrieve the source files from hgweb. I did a lot of searching, checking, fixing etc.

It just so happens that I lucked out bigtime, as I finally noticed that the symbols being loaded when I started up Minefield weren't coming from my symbol server. Instead, the symbols being loaded were symbols inside my object directory. I'm not sure when these symbols got created but they were scattered throughout the object directory, and unlike the symbols I was using, they weren't indexed. So, to test this out I deleted the xul.pdb in the object directory and re-attached to Minefield. Success! Now xul.dll was loading the pdb from my server instead of the objdir one, the source code was being pulled from hgweb and all was well.

Now, I'm not sure what my deliverable should be. Perhaps it should just be the new pdb's I've written, or the new symbolstore.py even. Or, maybe I've done enough to submit a patch.

EDIT: I have filed a patch under Bug 440001

0.2 Release - Lots of Progress

2008-11-12T17:56:00.004-05:00

Firstly, to add to my last post I decided to set my deliverable for 0.2

Hack symbolstore.py to index the pdb files

Serve the pdb files over my server and make sure they successfully retrieve source code from hgweb

Well, I'm happy to report that I've managed to accomplish the first of these tasks. A big thank you to lsblakk and ted for their help.

After reading up on Python, I came to understand a lot more about how symbolstore.py works.

Basically, it parses out a list of source files from each .sym file, writes a .stream file using the list of source files and calls pdbstr to write the data block to the pdb files.

Fortunately for me, lsblakk's previous modifications to symbolstore.py to make it work with CVS gave me all of the code I needed. All that was required from me was to make some slight changes.

Firstly, I needed to change the SourceIndex() method to write my data block instead of the one used previously for CVS. My new method is listed below:


def SourceIndex(fileStream, outputPath, hg_root):
    """Takes a list of files, writes info to a data block in a .stream file"""
    # Creates a .pdb.stream file in the mozilla\objdir to be used for source indexing
    # Create the srcsrv data block that indexes the pdb file
    result = True
    pdbStreamFile = open(outputPath, "w")
    pdbStreamFile.write('''SRCSRV: ini ------------------------------------------------\r\nVERSION=2\r\nINDEXVERSION=2\r\nVERCTRL=http\r\nSRCSRV: variables ------------------------------------------\r\nHGSERVER=http://hg.mozilla.org/mozilla-central''')
    # pdbStreamFile.write(hg_root)
    pdbStreamFile.write('''\r\nSRCSRVVERCTRL=http\r\nHTTP_EXTRACT_TARGET=%hgserver%/index.cgi/raw-file/%var3%/%var2%\r\nSRCSRVTRG=%http_extract_target%\r\nSRCSRVCMD=\r\nSRCSRV: source files ---------------------------------------\r\n''')
    pdbStreamFile.write(fileStream) # can't do string interpolation because the source server also uses this and so there are % in the above
    pdbStreamFile.write("SRCSRV: end ------------------------------------------------\r\n\n")
    pdbStreamFile.close()
    return result

Secondly, I simply needed to change the ProcessFile method inside of the Dumper class to look for hg files instead of cvs files. So from:


if filename.startswith("cvs"):

To:


if filename.startswith("hg"):

Like I said, no crazy coding required.

I ran into problems when trying to run 'make buildsymbols'. For some reason empty pdb files were being created, so I concluded that symbolstore.py wasn't running properly. I eventually discovered that the makefile wasn't running symbolstore.py using the -i flag. This flag is what tells symbolstore.py to add source file information to the pdb files.

I discovered that there was actually code in the makefile to add this flag:


ifdef PDBSTR_PATH
MAKE_SYM_STORE_ARGS += -i
endif

So, the reason that the pdb files were empty was because PDBSTR_PATH wasn't being set, so I simply added it to my environment variables. After that, voila! The pdb files were successfully indexed upon creation.

My next step is to serve these pdb files from a server and make sure that the source files are being retrieved from hgweb. I don't think I can do this with a localhost server because the debugger will find my local source files before it tries to look on hgweb. So, I need to find a place to serve my pdb files (I would like to find a remote server to do it so that I can maybe offer a way for people to test it).

0.2 Release Update

2008-11-06T22:23:00.003-05:00

I've been working on some preliminary tasks in preparation for my 0.2 release. These tasks include learning more about Python and learning more about how symbolstore.py works. So far I think I have a decent grasp of both.

I'm thinking my deliverable for this release will be an updated version of symbolstore.py. I'm not sure if 0.2 is going to be able to solve the full problem or if it will simply solve a portion of it, but I will try to shoot for something that indexes all of the source files.

Stay tuned.

FSOSS Report

2008-11-03T21:45:00.002-05:00

My paper has been finished, you can find it here. Overall FSOSS was a very interesting experience, and I definitely enjoyed it. I especially enjoyed the Mozilla and Mobile, Komodo and Open Source Design presentations.

Lab 7: Modifying Firefox

2008-11-02T09:39:00.003-05:00

This was a pretty interesting lab. I really enjoyed being able to see my own changes to the Firefox code affecting the actual browser.

It required me to dig a little deeper into the code than I am used to, fortunately MXR is always there to help me out with that.

Using the recommended changes from the wiki, I found that there were still some bugs. There was a bug when trying to delete a tab when there was less than 3, and also when I tried to create a new tab when there was only one open it would create 2 instead of 1.

Mercurial Source Server Integration 0.1 Released!

2008-10-18T17:55:00.004-04:00

You (the reader of course) will have to forgive my lack of updates on the 0.1 release, it's been a busy week with studying for midterms and whatnot. However, the good news is my 0.1 release has been completed. For instructions about how to set up the release, see this page

I guess since I never officially listed my objectives for this release I'll do it now.

Set up existing source server to see how it works(will download pdb's off mozilla server)

Set up localhost server

Build firefox and create pdb files (symbols), host them on local server

Insert my own data block into one of the mozilla pdb files (will use xul.pdb)

Get xul.pdb to grab source files off of hgweb instead of cvs repo

It was a long process trying to accomplish these tasks. The most daunting was trying to properly insert my data block into xul.pdb and have it work.

First I should point out that I had 2 locations for symbols set up. The first was a folder that stored the symbols i downloaded from http://symbols.mozilla.org/firefox (ill refer to this as source 1), the second was a folder that contained symbols i generated myself using make buildsymbols on my local version of minefield that I grabbed from hg's mozilla-central tree. I then copied these symbols to my local IIS server (ill refer to this as source 2).

My initial goal was to modify the xul.pdb in source 2 to grab the source files off of hgweb. To do this, I had to insert my own data block into xul.pdb. The data block is used to tell the symbol where to retrieve it's source files. Here is the data block I created


SRCSRV: ini ------------------------------------------------
VERSION=2
INDEXVERSION=2
VERCTRL=http
SRCSRV: variables ------------------------------------------
HGSERVER=http://hg.mozilla.org/mozilla-central
SRCSRVVERCTRL=http
HTTP_EXTRACT_TARGET=%hgserver%/index.cgi/raw-file/7356c512e9e1/%var3%/%fnfile%(%var1%)
SRCSRVTRG=%http_extract_target%
SRCSRVCMD=
SRCSRV: source files ---------------------------------------
e:\fx19rel\winnt_5.2_depend\mozilla\xpcom\io\SpecialSystemDirectory.cpp*MYSERVER*xpcom/io
e:\fx19rel\winnt_5.2_depend\mozilla\xpcom\io\SpecialSystemDirectory.h*MYSERVER*xpcom/io
e:\fx19rel\winnt_5.2_depend\mozilla\netwerk\cache\src\nsCacheMetaData.h*MYSERVER*netwerk/cache/src
e:\fx19rel\winnt_5.2_depend\mozilla\netwerk\cache\src\nsCacheService.h*MYSERVER*netwerk/cache/src
e:\fx19rel\winnt_5.2_depend\mozilla\widget\src\xpwidgets\nsBaseAppShell.h*MYSERVER*widget/src/xpwidgets
e:\fx19rel\winnt_5.2_depend\mozilla\widget\src\xpwidgets\nsBaseAppShell.cpp*MYSERVER*widget/src/xpwidgets
SRCSRV: end ------------------------------------------------

This data block was hard to nail down, because there was almost NO information on the web in regards to how to properly use http to extract source files. I hope that my blog can become a valuable resource in that regard.

The key is the variables section, the HTTP_EXTRACT_TARGET var tells the pdb where to grab the source file from. You'll notice keywords such as %var3%, this is referring to the lines in the source files section, where each var is seperated by an *. So for the first source file line, var3 is xpcom/io. You've probably noticed that I don't really need var1 and var2 in this instance, you're right. This data block was a modification of the existing data block in source 1's xul.pdb, i just took it and modified it.

Using the pdbstr.exe found in the Debugging Tools for Windows, I wrote the data block to source 2's xul.pdb. I ran srctool -f on the pdb to see if it was indexed, the response was that no files were indexed. I was suprised at this. When I tried to analyze the code in symbolstore.py (which I will have to modify in later releases), I came to the conclusion that the script simply gathered all the source file paths, wrote them to an auto-generated data block, and then wrote that data block to pdbstr. So when I tried to use this method manually, I was suprised to find out that I was unsuccessful. Perhaps my data block wasn't written properly.

I tried to test the data block instead on source 1. When I ran srctool on source 1's xul.pdb after inserting my own data block, it showed me that 6 files were indexed, and you'll noticed that there are 6 source files in the data block. To further check, I re-ran Visual Studio, and re-attached to firefox with source 1 as my source server (see the instructions above for further information on the process). Success! Clicking on the call to nsBaseAppShell.cpp in the Call Stack downloaded the source file from hgweb.

I still have a lingering question. Why was the source 1's xul.pdb able to index/work with my data block, while source 2's xul.pdb wasn't? I'm thinking that perhaps something is done to mozilla's symbols that I'm not aware of. Either I missed something in symbolstore.py or something happens outside of symbolstore.py that indexes the files.

Another observation I had was in regards to using Minefield. When debugging Minefield using source 2, all the source files were downloaded from my own machine. I guess the way pdb's work is that they first check for a local copy of the source file, and if it's not there they make the call to whatever remote site is specified in the data block.

My next objective for 0.2 is still to be determined. I'd also like to thank ted and lblakk for all the help they've given me with various aspects of this release.

Just one final note: My style has traditionally been to leave things alone and finishing them later (or to put it bluntly, procrastinating). I've learned through doing this release that for a project like this, that won't fly. Between having to study for midterms this week and doing this release, I feel lucky to have gotten everything done. For 0.2 and beyond, I plan on getting an early jump on things. That means smaller, more frequent blog posts instead of the essay I'm typing right now.

My biggest obstacle with this project so far was just getting started, because I absolutely knew nothing about it. Now that I've come to understand certain aspects of it, I feel a lot more enthusiastic about it, and I hope my future work reflects that enthusiasm.

Working Towards my 0.1

2008-10-13T13:10:00.005-04:00

I'm not exactly sure what I'll need to do to complete my 0.1, however one thing I know I need to do is get this source server working in CVS so that I can get a better idea of how it needs to work with Mercurial.

Getting the symbols to load was easy enough. All I had to do was add the location of Mozilla's symbols to Visual Studio.

Getting the source server to work properly has been slightly more tricky. Although it seemed easy enough in the Instructions I've been unable to retrieve the source files. My errors for the source files look like this:

SRCSRV: cvs.exe -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot checkout -r 3.68 -d 3.68 -N mozilla/widget/src/windows/nsAppShell.cpp
SRCSRV: Source server cannot retrieve the source code for file 'e:\fx19rel\winnt_5.2_depend\mozilla\widget\src\windows\nsappshell.cpp' in module 'C:\Program Files\Mozilla Firefox\xul.dll'. The system cannot find the file specified.

It seems like I'm close enough to getting it to work, so I'm not worried. My biggest worry at the moment is not knowing where to start to get this thing working in Mercurial.

One clue I have is from downloading the Debugging Tools for Windows. Inside the srcsrv folder is a bunch of .PM files (perl modules), including ones for cvs and svn. Using a Document I found on Lukas's blog, I found this piece of info:

The .PM files are the Perl modules that can talk to the version control systems; the SSINDEX.CMD loads these modules and calls through a standard interface to isolate the system differences. The other CMD files, such as VSSINDEX.CMD, are wrappers for working with specific version control systems, as SSINDEX.CMD needs the version control system on the command line.

Does this mean I need to create a .PM file for Mercurial? I'm hoping more talks with Ted and Lukas will bring that to light.

Lab 6: Thunderbird Bux Fix Lab

2008-10-11T14:11:00.003-04:00

I found that building Thunderbird was easy enough. The challenge with this assignment was finding a regular expression that could properly match all of the following email address formats:

Eventually I realized that finding a regular expression was pointless since I wouldn't be able to actually use it, so I tried thinking of general rules that would make an email address invalid:

1. Periods can't be right next to eachother
2. An address can't end in a period
3. An address can't begin with an @
4. An address can't have a period right after the @

The next step was to find the relevant code. Using the hint given to us by Dave, I simply did a find on the word mailto and came across line 198 of the mozTXTToHTMLConv.cpp

if (inString.FindChar('.', pos) != kNotFound)
{
aOutString.AssignLiteral("mailto:");
aOutString += aInString;
}

This seemed to be the block of code that evaluated the e-mail address. I noticed that it only checked for a . right after the @ sign, so using the rules I established for a valid e-mail, I changed the if statement to this:

if (inString.FindChar('.', pos) != kNotFound && inString.Find("..", 0) == kNotFound && inString.CharAt(inString.Length() - 1) != '.')
{
aOutString.AssignLiteral("mailto:");
aOutString += aInString;
}

Now the if checks for a '.' right after the @, 2 '.' next to eachother, and a '.' in the last character of the string.

Still In Progress....

Lab 2: Firefox Build Lab

2008-10-07T23:13:00.010-04:00

I know this is quite late, however I felt it would be better to at least do this lab and blog about it later rather than never.

What was hard?

I didn't find anything about building Firefox too hard. I had everything I needed to know in the documentation. I guess I could say that the trickiest part was figuring out what I needed to include in my .mozconfig file, but in the end this didn't cause me too much trouble.

What did you learn that you didn't know before?

Where do i start? I had the chance to use software I hadn't touched before in my life such as Mercurial and Minefield. I discovered how mozilla's build system worked and what was needed to grab, setup and run that build. Pretty much every aspect of this lab was something I had never done before.

How would you do it differently next time?

I don't think I could've approached this lab any better than how I did. I didn't really have any problems, I read the documentation thoroughly, so I don't think I really needed to do anything differently to be successful. I think if I ever built Firefox again it would be within Linux, as I've heard that it builds faster there than it does in Windows.

What advice would you give others trying this?

Read the instructions carefully. I had heard of others having problems performing this lab so I made sure that when I did it (finally) that I understood what was in the documentation. Because of that, I was able to complete the lab without any problems

Here is a screenshot of my build:

I know I haven't been a model student with these labs, however I fully intend to go back and complete the labs I've missed so I can be more prepared when doing my project (which I will also have updates on soon).

Note: I also made sure to submit my build times to Patrick Lam's wiki page.

A Little Late But...My First Post

2008-09-21T11:36:00.003-04:00

So, I've finally gotten around to making a blog post, better late then never I suppose.

I've been thinking about my potential project for the last couple of days now, I've narrowed my choices down to two: adding source server support to Mercurial and creating a self-serve symbol upload system.

On the one hand, I've been leaning towards web development for the last year or so, so creating a web-based system definitely appeals to me and would allow me to perhaps brush up on my PHP skills(although I'm not entirely sure how I would approach the project yet). On the other hand, taking the Mercurial project would give me a great resource in Lukas(who worked on the original project) as well as ted, who I have yet to talk to. Also, the Mercurial project sounds like a more important project at first glance, although I can't be too sure about that.

Hopefully I'll be able to make a decision within the next couple of days, in the mean time I'll be updating this blog soon with my lab results.

Test

2008-09-11T12:41:00.001-04:00

this is a test post