Showing posts with label El Capitan. Show all posts
Showing posts with label El Capitan. Show all posts

Wednesday, October 26, 2016

pyenv + pyenv-virtualenv + OpenCV 3 on Mac OS X El Capitan

Python is a good tool for quick project startup while it's still a little early to dig into the details of C++ or Objective C code.

The trouble is that the builtin Python version may not meet our needs. As you know, it starts to get messy when a couple of different versions of Python interpreters have been installed in the same OS. A few weeks later, you may find out things are not compiling properly due to all sorts of PYTHONPATH issues.

So, here comes the management tool - pyenv

Simply speaking, pyenv lets you easily switch between multiple versions of Python. On OS X, pyenv may install multiple versions of Python into its own repository location while the builtin version of Python is still intact with the OS. This minimises the conflict between your project and other existing applications which uses Mac's builtin Python for compiling work.

pyenv also comes with a plugin to deal with Python's virtual environment - pyenv-virtualenv 

pyenv-virtualenv provides features to manage virtualenvs and conda environments for Python.

With both of these, you can customise a particular version of Python as per Application based the virtual environment will trigger those changes for individual project folder.

To setup, please read through the README page via the following links:

To keep things simple, it is suggest that we should use homebrew to install above packages.

In the file ~/.bash_profile, typical setup is recommended as follows:

#
#
# Multi-Python switcher Pyenv
# To use Homebrew's directories rather than ~/.pyenv add to your profile:
export PYENV_ROOT=/usr/local/var/pyenv
export PATH="$PYENV_ROOT/bin:$PATH"
# Enable shims and autocompletion
if which pyenv > /dev/null; then eval "$(pyenv init -)"; fi

# A pyenv plugin to manage virtualenv (a.k.a. python-virtualenv) ref: https://github.com/yyuu/pyenv-virtualenv
# Automatically activate/deactivate virtualenvs on entering/leaving directories 
# which contain a .python-version file that lists a valid virtual environment
eval "$(pyenv virtualenv-init -)"
export PYENV_VIRTUALENV_DISABLE_PROMPT=1

# pip should only run if there is a virtualenv currently activated
export PIP_REQUIRE_VIRTUALENV=true
#
#
#
#


OpenCV is an excellent library for image/video processing. To avoid any error during the installation, it's recommended we install via homebrew command:


$
# First time installation
$ brew install opencv3 --HEAD --with-python3 --with-ffmpeg --with-tbb --with-contrib --with-opengl --with-qt5
# Upgrade alternative
$ brew reinstall opencv3 --HEAD --with-python3 --with-ffmpeg --with-tbb --with-contrib --with-opengl --with-qt5
$

All these will be done at global system level.

Now, using command like 'pyenv virtualenv' to setup a virtual environment for your project:
# Get into empty target project folder
$ mkdir ~/target_project
$ cd ~/target_project
$
# List available Python versions
$ pyenv versions
  system
  2.7.10
  2.7.12
  3.5.2
# Create new virtual environment with specific version
# Also name it 'my-env-3.5.2' or any other name
$ pyenv virtualenv 3.5.2 my-env-3.5.2
# Now activate new virtual environment within the target project folder
$ pyenv activate my-env-3.5.2
$

Assuming OpenCV 3 and Python 3.5.2 will be used within new virtual environment, it's time to link up OpenCV library folder to the target Python folder like /site-packages for importing reference.
# Assuming in ~/target_project folder
$ echo `brew --prefix opencv3`/lib/python3.5/site-packages >> $PYENV_ROOT/versions/my-env-3.5.2/lib/python3.5/site-packages/opencv3.pth
$



For Python virtual environment, we can setup necessary packages by using the following command within the target project folder

# Assuming in ~/target_project folder
$ 
$ pip install numpy
$

To test whether OpenCV 3 is supported within the virtual environment, let's open up Python console and import cv2 library:

# Assuming in ~/target_project folder
$ python
Python 3.5.2 (default, Jul 19 2016, 15:25:16) 
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>import cv2
>>>

If no error is generated, we can ensure that OpenCV 3 library support is in place with current Python environment.




Wednesday, May 11, 2016

Practical way to tackle WiFi dropout and slow performance issue in OS X El Capitan

There is a couple of occasions that I need to do have an upgrade on Mac OS X. Chances are I completely forget those tweaking tips and hope things will be resolved in next version. Something like WiFi connection tweaking I can remind from the forum. An explanation can help me remind those things I should do every single time when I need to do an OS X upgrade.

We can find all kinds of topics regarding WiFi dropout issue on OS X and someone may suggest standard procedure to get around this. For WiFi settings, Apple would like to stick to standard MTU size at 1500 bytes for Ethernet network. But, why?

With WiFi connection, we can do a ping test like this:

$
$
$ ping -D -s 1500 google.com
PING google.com (203.5.76.246): 1500 data bytes
ping: sendto: Message too long
ping: sendto: Message too long

Note:
Option -D suppresses fragmentation for the packet (force to transmit whole packet at a time)
Option -s 1500 specify the packet size, i.e., 1500 bytes

Problem is there is an overhead added on top of packet size for every transmission. That is why we get 'Message too long' responses from there.

To rectify this problem, let's do the math:

Optimum MTU size = Packet size + Overhead

What about the overhead? The overhead would be 28 bytes because 20 bytes are reserved for the IP header and 8 bytes must be allocated for the ICMP Echo Request header.

So, maximum packet size permitted in each network packet is actually:

Optimum MTU size = Non-fragmented Packet size +  28 bytes

Here we say MTU size will theoretically be 1500. Packet size allowed would be

(1500 -28)=1472 bytes

But, is it practical? Let's test this.

To test this magic number, issue the following command in Terminal:

$
$
$ ping -D -s 1472 google.com
PING google.com (203.5.76.246): 1472 data bytes
1480 bytes from 203.5.76.246: icmp_seq=0 ttl=56 time=7.787 ms
1480 bytes from 203.5.76.246: icmp_seq=1 ttl=56 time=4.838 ms
1480 bytes from 203.5.76.246: icmp_seq=2 ttl=56 time=4.360 ms
1480 bytes from 203.5.76.246: icmp_seq=3 ttl=56 time=6.184 ms

Packets are now transmitted successfully with this payload (1472). Therefore, we can be sure the optimum MTU payload size would be 1500 (luckily a theoretical value this time) just for this particular WiFi network.

With no fragmentation happening, each packet should be theoretically transmitted as a whole and not broken in parts. This can improve the network speed effectively in an ideal signal condition.

Just remind that the size of payload can be different across various locations and wireless routers. You might need to find the smallest optimum payload which sits within the size range of successful transmission for most networks you'll be connecting, either at home or office. Try stepping down the value until you find transmission works for all the networks you have tried.

Apply this payload value in Network settings panel and hopefully it'll help reduce the number of WiFi dropout and improve the WiFi network speed.












Sunday, October 4, 2015

Fix Hombrew problems after upgrading Mac OS X from Mavericks to El Capitan, skipping Yosemite

I have been sticking with Mavericks (OS X 10.9.x) since released and now skip Yosemite and directly upgrade to the solid build of El Capitan (OS X 10.11). It's just a direct upgrade and things seem to be working after a couple of reboots.

Well, we all know installed packages may not work at all. For instance, Developer tools like Homebrew seems broken again.

As seen on the Github, people have started collaborating with each others to resolve this.

https://github.com/Homebrew/homebrew/issues/40519

First thing to fix broken Homebrew directory is the permission issue. Error like this for brew update command:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
Stashing your changes:
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
Error: Failure while executing: git stash save --include-untracked --quiet


So, try the following command first:

sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local

Second trial of brew update, another error appeared while doing brew update as follows:

Error: Failure while executing: git pull -q origin refs/heads/master:refs/remotes/origin/master.

Since major upgrade, we need to install recent release of Xcode Command Line Tools. An easy way is to install Xcode 7.0.1 from AppStore and then open Xcode to agree the terms and conditions.

Within CXcode, click menu item:

[Xcode]->[Open Developer Tool]-> [More Developer Tools...]

It helps to open Apple's download website for Developer Tools.

Click to download and install the following package :

Command Line Tools OS X 10.10 for Xcode 7

Reopen Terminal again and then try brew update, once again another error message shows up. But this time it looks like things get fixed by the second call of brew update command. So, issue the update command twice to get itself fixed and ready:

$brew update
#Error message...
$brew update
#It works this time...