0.2 - Finished

Well, after a lot of trial and error I got my local server to serve the pdb files properly.

The big problem I was having was that although the symbols were being loaded, when I tried to retrieve source code it would only ask me to locate my local source files and not try to retrieve the source files from hgweb. I did a lot of searching, checking, fixing etc.

It just so happens that I lucked out bigtime, as I finally noticed that the symbols being loaded when I started up Minefield weren't coming from my symbol server. Instead, the symbols being loaded were symbols inside my object directory. I'm not sure when these symbols got created but they were scattered throughout the object directory, and unlike the symbols I was using, they weren't indexed. So, to test this out I deleted the xul.pdb in the object directory and re-attached to Minefield. Success! Now xul.dll was loading the pdb from my server instead of the objdir one, the source code was being pulled from hgweb and all was well.

Now, I'm not sure what my deliverable should be. Perhaps it should just be the new pdb's I've written, or the new symbolstore.py even. Or, maybe I've done enough to submit a patch.

EDIT: I have filed a patch under Bug 440001

0.2 Release - Lots of Progress

Firstly, to add to my last post I decided to set my deliverable for 0.2

• Hack symbolstore.py to index the pdb files


• Serve the pdb files over my server and make sure they successfully retrieve source code from hgweb

Well, I'm happy to report that I've managed to accomplish the first of these tasks. A big thank you to lsblakk and ted for their help.

After reading up on Python, I came to understand a lot more about how symbolstore.py works.

Basically, it parses out a list of source files from each .sym file, writes a .stream file using the list of source files and calls pdbstr to write the data block to the pdb files.

Fortunately for me, lsblakk's previous modifications to symbolstore.py to make it work with CVS gave me all of the code I needed. All that was required from me was to make some slight changes.

Firstly, I needed to change the SourceIndex() method to write my data block instead of the one used previously for CVS. My new method is listed below:

def SourceIndex(fileStream, outputPath, hg_root):
    """Takes a list of files, writes info to a data block in a .stream file"""
    # Creates a .pdb.stream file in the mozilla\objdir to be used for source indexing
    # Create the srcsrv data block that indexes the pdb file
    result = True
    pdbStreamFile = open(outputPath, "w")
    pdbStreamFile.write('''SRCSRV: ini ------------------------------------------------\r\nVERSION=2\r\nINDEXVERSION=2\r\nVERCTRL=http\r\nSRCSRV: variables ------------------------------------------\r\nHGSERVER=http://hg.mozilla.org/mozilla-central''')
    # pdbStreamFile.write(hg_root)
    pdbStreamFile.write('''\r\nSRCSRVVERCTRL=http\r\nHTTP_EXTRACT_TARGET=%hgserver%/index.cgi/raw-file/%var3%/%var2%\r\nSRCSRVTRG=%http_extract_target%\r\nSRCSRVCMD=\r\nSRCSRV: source files ---------------------------------------\r\n''')
    pdbStreamFile.write(fileStream) # can't do string interpolation because the source server also uses this and so there are % in the above
    pdbStreamFile.write("SRCSRV: end ------------------------------------------------\r\n\n")
    pdbStreamFile.close()
    return result

Secondly, I simply needed to change the ProcessFile method inside of the Dumper class to look for hg files instead of cvs files. So from:

if filename.startswith("cvs"):

To:

if filename.startswith("hg"):

Like I said, no crazy coding required.

I ran into problems when trying to run 'make buildsymbols'. For some reason empty pdb files were being created, so I concluded that symbolstore.py wasn't running properly. I eventually discovered that the makefile wasn't running symbolstore.py using the -i flag. This flag is what tells symbolstore.py to add source file information to the pdb files.

I discovered that there was actually code in the makefile to add this flag:

ifdef PDBSTR_PATH
MAKE_SYM_STORE_ARGS += -i
endif

So, the reason that the pdb files were empty was because PDBSTR_PATH wasn't being set, so I simply added it to my environment variables. After that, voila! The pdb files were successfully indexed upon creation.

My next step is to serve these pdb files from a server and make sure that the source files are being retrieved from hgweb. I don't think I can do this with a localhost server because the debugger will find my local source files before it tries to look on hgweb. So, I need to find a place to serve my pdb files (I would like to find a remote server to do it so that I can maybe offer a way for people to test it).

0.2 Release Update

I've been working on some preliminary tasks in preparation for my 0.2 release. These tasks include learning more about Python and learning more about how symbolstore.py works. So far I think I have a decent grasp of both.

I'm thinking my deliverable for this release will be an updated version of symbolstore.py. I'm not sure if 0.2 is going to be able to solve the full problem or if it will simply solve a portion of it, but I will try to shoot for something that indexes all of the source files.

Stay tuned.

FSOSS Report

My paper has been finished, you can find it here. Overall FSOSS was a very interesting experience, and I definitely enjoyed it. I especially enjoyed the Mozilla and Mobile, Komodo and Open Source Design presentations.

Lab 7: Modifying Firefox

This was a pretty interesting lab. I really enjoyed being able to see my own changes to the Firefox code affecting the actual browser.

It required me to dig a little deeper into the code than I am used to, fortunately MXR is always there to help me out with that.

Using the recommended changes from the wiki, I found that there were still some bugs. There was a bug when trying to delete a tab when there was less than 3, and also when I tried to create a new tab when there was only one open it would create 2 instead of 1.