Recent Posts

OOP

How to contribute to an open source project

I understand. I really do. I get nervous too when thinking about submitting the “ifs” and the “elses” to an existing open source project. Finding the time and the courage to submit code to an open source project is still difficult for me. Here are some opinions of mine that I offer to you. Hopefully it will help you along.

Find a problem

This one is simple. Really it is. The next time you are using some code that is open source and you happen to run into a bug, odd behavior, or some needed functionality, take some time and figure out if you have the ability to fix it or add some value to the project.

I recently found some strangeness in Guard. It was running my tests twice and after about fifteen minutes of work, I finally came to the conclusion that I needed to understand what was happening.

I spent some time tracing what Guard was doing and realized that if my Guardfile had duplicate definitions, it would run for every one present. For me, this was no bueno.

Evaluate the community

I looked at the prior contributions and the community members. Briefly stalking them on teh githubz and teh twitterz to see if they were nice in how they engaged others. It appeared they were.

Know your skillz

I cloned the Guard code base and took a look around. I could see that it was a solid project but it didn’t smell of elitism. I felt that my talent was up to the task of contributing as it didn’t appear that super-duper-primo pull requests were required right out of the gate. I prefer this, because you can get your ideas merged, and then refactor them later once the idea is vetted.

Validate the issue doesn’t exist

My buddy Matthew Turland points out that newbies should take the time to check issues/PRs to ensure that a fix hasn’t already been submitted as you don’t want to duplicate work.

This basic project reconnaissance helped tell me that my potential contribution, if not included, would be at the very least appreciated.

Be a polar bear

When it comes to the code I submit, I’m totally open to “eating my young” and assume that any code I write is just one turd shy of making a full toilet. Detaching myself from any emotion concerning my hackery will leave me in a much better position to grow.

Don’t be a douche

It may go without saying, but here it is again. When submitting a bug or pull request, I always try to be polite. Even if upon my communication, I realize that the community I’ve engaged is completely full of dicks. At that point, I politely exit the conversation and never go back to the project. I don’t Facebook or Twitter the community’s douchery. Seriously, humbleness is a virtue here and is a better approach. My time is better spent on finding a community that will accept me, rather than hating one that won’t.

Follow community guidelines

Since this is my first interaction with @thibaudgg and Guard, I went into a lot of detail with the issue. I tend to do this by default. But it helps articulate to the community that I spent time understanding and verifying what I’m trying to accomplish.

Value proposition

In my submission (pull request) I included what I felt the problem was right at the very top of the post. But I concluded with the value proposition. Articulating that by including my feature, it would help the project achieve (X).

Embracing failure

In this case, my code was appreciated and accepted. Which is a great feeling when you get to be included in something. Even when the contribution is small.

However, when I get rejected, I tend to ask for suggestions on what it would take to get my code or idea included. Asking for advice or help is surprisingly a skill that many people suck at. I can only assume because many people’s egos don’t allow for them to be wrong. Too bad for them. Because by embracing my failure I get a world of value out of it in the form of friendship and learning. People will willing teach me things for the incredibly cheap price of simply admitting I don’t know the correct answer and then asking for their help.

In conclusion

It is the continual hustle in the face of rejection that needs encouragement, especially when you strike out. Track down the people that praise the hustle. And if you need some support find me on twitter and I’ll give you a high five and do whatever else I can to further your success. I’m happy to help.

 

(Nerd) Mechanize & Javascript

This is from the mechanize site, wish I would have read it before I started.

Since Javascript is completely visible to the client, it cannot be used to prevent a scraper from following links. But it can make life difficult, and until someone writes a Javascript interpreter for Perl or a Mechanize clone to control Firefox, there will be no general solution. But if you want to scrape specific pages, then a solution is always possible.

One typical use of Javascript is to perform argument checking before posting to the server. The URL you want is probably just buried in the Javascript function. Do a regular expression match on $mech->content() to find the link that you want and $mech->get it directly (this assumes that you know what you are looking for in advance).

In more difficult cases, the Javascript is used for URL mangling to satisfy the needs of some middleware. In this case you need to figure out what the Javascript is doing (why are these URLs always really long?). There is probably some function with one or more arguments which calculates the new URL. Step one: using your favorite browser, get the before and after URLs and save them to files. Edit each file, converting the the argument separators (‘?’, ‘&’ or ‘;’) into newlines. Now it is easy to use diff or comm to find out what Javascript did to the URL. Step 2 – find the function call which created the URL – you will need to parse and interpret its argument list. Using the Javascript Debugger Extension for Firefox may help with the analysis. At this point, it is fairly trivial to write your own function which emulates the Javascript for the pages you want to process.

Here’s annother approach that answers the question, “It works in Firefox, but why not Mech?” Everything the web server knows about the client is present in the HTTP request. If two requests are identical, the results should be identical. So the real question is “What is different between the mech request and the Firefox request?”

The Firefox extension “Tamper Data” is an effective tool for examining the headers of the requests to the server. Compare that with what LWP is sending. Once the two are identical, the action of the server should be the same as well.

I say “should”, because this is an oversimplification – some values are naturally unique, e.g. a SessionID, but if a SessionID is present, that is probably sufficient, even though the value will be different between the LWP request and the Firefox request. The server could use the session to store information which is troublesome, but that’s not the first place to look (and highly unlikely to be relevant when you are requesting the login page of your site).

Generally the problem is to be found in missing or incorrect POSTDATA arguments, Cookies, User-Agents, Accepts, etc. If you are using mech, then redirects and cookies should not be a problem, but are listed here for completeness. If you are missing headers, $mech->add_header can be used to add the headers that you need.

LINK

(Nerd) Ubuntu + Vim + ZendFramework-1.9.2 + .pthml syntax highlighting

1) From the command line ‘cd’ to your ‘home’ directory


test-box@jbuntu:~$ cd

2a) Check to see if the .vimrc file exists


test-box@jbuntu:~$ ls .vimrc

If the terminal outputs nothing, then that means the file does not exist.

2b) If you get the following


test-box@jbuntu:~$ ls .vimrc
.vimrc

It means that the file does exist and we just need to edit it.

3) If the file does not exist just ‘touch’ the file. If it DOES exist, just skip this step.


test-box@jbuntu:~$ touch .vimrc

4) From this point, ‘vi’ the ‘.vimrc’ file. You primarily need the following lines and you are more than welcome to copy/paste. Write/Quite when finished.


if has("autocmd")
autocmd BufEnter *.phtml set syn=php
endif
syn on

Now the next time you open VI it should have the desired highlighting for .phtml files found in the Zend Framework.

Below is my ‘.vimrc’ file in it’s entirety. Just for the record. It also allows for syntax highlighting to occur in CakePHP .ctp files along with some other settings that I prefer.


set tabstop=2
set shiftwidth=2
set expandtab
if has("autocmd")
autocmd BufEnter *.ctp set syn=php
autocmd BufEnter *.phtml set syn=php
endif
syn on
set ai

Let my dataset change your mindset

Posted on August 30, 2009

If you follow the link provided. It will take you to a brilliant video by Hans Rosling. My hope, is that anyone watching at least begins to understand the power of data. Maybe not the exact method to manipulate the stuff, but at least the desire to know more about it.

This video is quite timely for me. For I have thought long and hard about putting together a series of posts on data analysis. Particularly on the work I have done in looking at the Deschutes County Clerks Data.

For those of you who don’t know. In the fall of 2007 I created a program that went to the county website and would pull public records and sales data concerning housing. It would throw this data into a usable database.  And I would use this data to educate family and friends on when our housing bubble was started, and how bad it really was.

I built a small website with this data, but stopped my work on it because of several complex reasons.

Anyway, if people are interested in how to analyze data, or would appreciate an ongoing conversation about it, let me know.

Older Posts