New projects! – Python, Selenium

I’ve been working with Python and Selenium recently at work. Got a few projects completed:

  • Automated a daily site checklist for www.recruiter.com
    • Inputs two csv lists of pages to check, one for authenticated users and another for unauthenticated
    • Checks all pages in the unauthenticated list
    • Records all meta tags, page titles, and h1 tags in an output csv
    • Takes a screenshot of the page and saves to file
    • Logs into a user account
    • Repeats for all authenticated pages
  • Wrote a script that pulls a daily report
    • Logs into mysql database
    • Runs multiple queries on the database and saves output to csv file
  • Script that dynamically inserts data into an html template and uploads to site
    • Currently in progress; major work done completed, site integration to be completed
    • Inputs csv doc and html template, uses data to create html page

Also, I recently discovered a sql injection vulnerability in development code at work (and successfully exploited it! Neat!). Dev fixed the issue.

On a more personal note, a friend sent me an email this afternoon telling me about profile on rateyourmusic.com. He has an unbelieveable amount over entries there; he’s rated over 4,000 albums. Admitting that he’s a bit crazy about it, he made a comment that he was slowly copying all his entries into a spreadsheet (over the course of months). Wow! That’s not right!

I wrote up a write a quick script for him after work this evening. It crawls through all 162 pages of his account and pulls the data into a spreadsheet automatically. I’m actually quite happy about this; I feel like it’s the first project I’ve written that has had a real purpose. In the future, I’ll make some updates to this:

  • Take a username as an input so it can be used for other accounts
  • Pull additional datafields into the spreadsheet
  • Determine the amount of pages in the account on it’s own, rather than manually entering it

I guess he’ll have to find else to do in his free time now that he doesn’t have to copy data into a spreadsheet…

That project is uploaded to github here.

Leave a Reply

Your email address will not be published. Required fields are marked *