Tag Archives: python

Setting Up Selenium on RaspberryPi 2/3

Selenium is a great tool for Internet scraping or automated testing for websites. I personally use it for scrapping on dynamic content website in which the content is created by JavaScript routines. Lately, I also tried to run Selenium on Raspberry and found out that it is not easy to install all requirements. Here I like to share my commands to make things easier for you.

Here I like to give a simple run-down to install all requirements to make Selenium available on a Raspi. Basically, we install first Firefox, then Geckodriver and finally Selenium and we are ready to go.

Before start,  better to note that ChromeDriver does not support ARM processors anymore, therefore it is not possible to use Chromium with Selenium on Raspberry.

First, install system requirements. Update the system, install Firefox and xvfb (display server implementing X11);

sudo apt-get update
sudo apt-get install iceweasel
sudo apt-get install xvfb

Then, install python requirements. Selenium, PyVirtualDisplay that you can use for running Selenium with hidden  browser display and xvfbwrapper.

sudo pip install selenium
sudo pip install PyVirtualDisplay
sudo pip install xvfbwrapper

Hope everything run well and now you can test the installation.

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(1024, 768))
display.start()

driver = webdriver.Firefox()
driver.get('http://www.erogol.com/')
driver.quit()

display.stop()

 

 

Share

How to use Python Decorators

Decorators are handy sugars for Python programmers to shorten things and provides more concise programming.

For instance you can use decorators for user authentication for your REST API servers. Assume that, you need to auth. the user for before each REST calls. Instead of appending the same procedure to each call function, it is better to define decorator and tagging it onto your call functions.

Let's see the small example below. I hope it is self-descriptive.


"""
How to use Decorators:

Decorators are functions called by annotations
Annotations are the tags prefixed by @
"""

### Decorator functions ###
def helloSpace(target_func):
def new_func():
print "Hello Space!"
target_func()
return new_func

def helloCosmos(target_func):
def  new_func():
print "Hello Cosmos!"
target_func()
return new_func


@helloCosmos # annotation
@helloSpace # annotation
def hello():
print "Hello World!"

### Above code is equivalent to these lines
# hello = helloSpace(hello)
# hello = helloCosmos(hello)

### Let's Try
hello()

 

Share

Passing multiple arguments for Python multiprocessing.pool

Python is a very bright language that is used by variety of users and mitigates many of pain.

One of the core functionality of Python that I frequently use is multiprocessing module. It is very efficient way of distribute your computation  embarrassingly.

If you read about the module and got used, at some point you will realize, there is no way proposed to pass multiple arguments to parallelized function. Now, I will present a way to achieve in a very Pythonized way.

For our instance, we have two lists with same number of arguments but they need to be fed into the function which is pooling.

Here we have self cover code:

Share

Project Euler - Problem 12

As a 2 years researcher, I feel a bit rusty to code.  I search a good set of execises to hone my abilities again and I stumbled upon Project Euler. This site hosts increasing number of very well formed algorithmic problems and discussions. It ranges very basic problems to very high level ones, requiring profound knowledge and practice.

After that intro. I want to introduce one of the example question from Project Euler. NOTE THAT, IF YOU ALREADY KNOW THE SITE AND YOU TRY TO SOLVE THAT PROBLEM, DO NOT CHEAT YOURSELF.

Here is the problem statement we try to solve. Continue reading Project Euler - Problem 12

Share

Two way python dictionary

Here there is a simple class implementation of Two way Dictionary that uses from native dictionary class of Python. The main idea of that kind of data structure to reach, but side of the data by using other side as the key. It is like bi-directional relation between items.

A dumb use case:

d = TwoWayDict()
d['erogol'] = 13
print d['erogol']
# outputs 13
print d[13]
# outputs erogol

Here is the class implementation. However, keep in mid that this class uses more x2 memory to keep the data with that functionality.

class TwoWayDict(dict):
    def __len__(self):
        return dict.__len__(self) / 2

    def __setitem__(self, key, value):
        dict.__setitem__(self, key, value)
        dict.__setitem__(self, value, key)

Hope this helps in some hinge 🙂

Share

Simple Parallel Processing in Python

Here is a very concise view of Python multiprocessing module and its benefits. It is certainly important module for large scale data mining and machine learning projects and Kaggle like challenges. Therefore take a brief look to that slide to discover how to up-up your project cycle.

For more info refer to :
Share

de-importing a Python module with a simple function

Sometimes it is eluding to de-importing some of the python modules especially dwelling on IPython since you have changes something on the module and IPython does not see those  changes even you import the module again. The solution of such a problem is to de-import the module and import again. Here is the function that de-import the staff.

 

 

Share