Using Stanford Phoenix++ MapReduce to count the most frequent words in Wikipedia Database

Log of a quick and fun afternoon project.

1. Download Wikipedia English-language Database:

http://en.wikipedia.org/wiki/Wikipedia:Database_download

Download pages-articles.xml.bz2

Uncompress it ro pages-articles.xml (around 45GB)

2. Download Phoenix++:

http://mapreduce.stanford.edu/

http://mapreduce.stanford.edu/plus/phoenix++-1.0.tar.gz

make binary file and then find ‘word_count’ in ‘tests’ folder

3. copy wordcount and pages-articles.xml to the same folder

./word_count pages-articles.xml 1000 >wikiwords.txt

Using wordcount program to count top 1000 popular words from wiki database

4. Result: (Using 8 Xeon E5 server, 128 processing cores) Continue reading Using Stanford Phoenix++ MapReduce to count the most frequent words in Wikipedia Database

Advertisements

Install github atom text editor on Ubuntu 14.04 x64

Official guide:

https://github.com/atom/atom/blob/master/docs/build-instructions/linux.md

Quote here:

  git clone https://github.com/atom/atom
  cd atom
  script/build # Creates application at /tmp/atom-build/Atom
  sudo script/grunt install # Installs command to /usr/local/bin/atom
  script/grunt mkdeb # Generates a .deb package at /tmp/atom-build

My log (for future re-installation): current version 0.95-f86280a

#**********************
#go to http://nodejs.org/download/
#download node.js source code
cd ~/tmp/nodejs
wget  http://nodejs.org/dist/v0.10.28/node-v0.10.28.tar.gz
tar xzvf node-v0.10.28.tar.gz
cd node-v0.10.28
./configure
./make test
#if failure occures at any stage, fix those. In my case, I failed at 'curl'.
sudo apt-get install curl
./make
sudo make install
#***********************
sudo apt-get install npm
sudo apt-get install libgnome-keyring-dev
sudo npm config set python /usr/bin/python2 -g
#*************************
cd ~/tmp/atom
git clone https://github.com/atom/atom
cd atom/script
sudo ./build
#err occurs at build: node-gyp, my solution:
sudo apt-get remove gyp
#This works
sudo ./build  #this will create app at /tmp/atom-build/Atom
#**************************
cd ~/atom/atom
sudo npm install -g grunt-cli
#don't know if above command is necessary or not
npm install
cd build
sudo grunt install #this will install atom to /usr/local/bin/atom
sudo grunt mkdeb #this will make .deb package at /tmp/atom-build
#****************************
#Finished!
#To run
sudo atom
#package management
apm

 

Windows 8.1 Event 10016 Fix: The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID {D63B10C5-BB46-4990-A94F-E40B9D520160}

A year ago, I had such error message in the event viewer on the first day of fresh installation of Windows 8. I believed it was also the reason that I had a blue screen: MEMORY_MANAGEMENT error. It was caused by ntoskrnl.exe upon inspecting Windows Mini Dump, by a tool named BlueScreenView.  Now a year later, after I recently updated to Windows 8.1 Update 1, the problem came back again. I decide to log it here.

After inspecting Windows Event Viewer, the error is the same as before, which is:

Event 10016, DistributedCOM

The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID
{D63B10C5-BB46-4990-A94F-E40B9D520160}
and APPID
{9CA88EE3-ACB7-47C8-AFC4-AB702511C276}
to the user NT AUTHORITYSYSTEM SID (S-1-5-18) from address LocalHost (Using LRPC) running in the application container Unavailable SID (Unavailable). This security permission can be modified using the Component Services administrative tool.

Here is a way I found to fix it:

1.Go to Component Service: dcomcnfg  (in Windows 8.1, you may hit ‘Win’ + Q and enter dcomcnfg, and launch it by Admin)

At the left panel, go to ‘Component Service->Computers->My Computer->DCOM Config’, then at the right panel, find by name: ‘RunTimeBroker’ (This is {D63B10C5-BB46-4990-A94F-E40B9D520160})

Right click it, choose ‘Property’ -> ‘Security’.   Then I found everything was grey and unable to edit.

2. To enable editing ‘Security’, launch regedit (‘Win’ + Q and enter regedit, open the application by Admin)

Browse to Hkey_classes_rootAppID{9CA88EE3-ACB7-47C8-AFC4-AB702511C276}, Right click it

Choose ‘Permissions’, -> Choose ‘Advanced‘ -> Change Owner to ‘Administrators’ of your PC -> Apply the change

Then change ‘SYSTEM’ and ‘Administrators’ ‘s permission to ‘Full Control’ under ‘Permission’ -> Apply the change

3. Go back to Component Service dcomcnfg

Now the Security section is able to edit.

In ‘Launch and Activation Permissions’, Add ‘SYSTEM’ and give it permission ‘Full Control’

4. DONE!

Continue reading Windows 8.1 Event 10016 Fix: The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID {D63B10C5-BB46-4990-A94F-E40B9D520160}

Putty Color Scheme Tango & My Putty Settings

I use Putty w/ Xming Server to replace  X-win 32 and VNC.

Color Scheme

After searching, I found a decent color scheme for Putty.

To make it default for Putty, paste the following code and name the file PuttyConfig.PS1  (Windows Power Shell Script):  (I borrow the script from another site, see links at the end)

param([string]$puttyConf)

$sessionKey = "HKCU:SoftwareSimonTathamPuTTYSessions$puttyConf"
if ( test-path $sessionKey ) {
   $values = @{
      "Xterm256Colour" = 0x00000001
      "BoldAsColour" = 0x00000001
      "Colour0" = "187,187,187"
      "Colour1" = "255,255,255"
      "Colour2" = "8,8,8"
      "Colour3" = "85,85,85"
      "Colour4" = "0,0,0"
      "Colour5" = "0,255,0"
      "Colour6" = "46,52,54"
      "Colour7" = "85,87,83"
      "Colour8" = "204,0,0"
      "Colour9" = "239,41,41"
      "Colour10" = "78,154,6"
      "Colour11" = "138,226,52"
      "Colour12" = "196,160,0"
      "Colour13" = "252,233,79"
      "Colour14" = "52,101,164"
      "Colour15" = "114,159,207"
      "Colour16" = "117,80,123"
      "Colour17" = "173,127,168"
      "Colour18" = "6,152,154"
      "Colour19" = "52,226,226"
      "Colour20" = "211,215,207"
      "Colour21" = "238,238,236"
   }
   $values.Keys | %{
      set-itemproperty $sessionKey $_ $values[$_]
   }
} else {
   write-error "Profile '$puttyConf' does not exist in the Putty configuration"
}

Fonts & Window

Double click the file to run it.  I also set Font to “Courier New”, 12-point, Font quality: ClearType

I wish to have a transparent putty terminal window, so I use a modified version of Putty: PuttyTray.  Then set window opacity( in Colours section ) to 220.

Export Putty Session Settings

To export putty settings with session list: (This will generate a .reg file on the Desktop)

regedit /e "%userprofile%desktopputty.reg" HKEY_CURRENT_USERSoftwareSimonTatham

To export sessions only:

regedit /e "%userprofile%desktopputty-sessions.reg" HKEY_CURRENT_USERSoftwareSimonTathamPuTTYSessions

Continue reading Putty Color Scheme Tango & My Putty Settings

Connect iPhone/Laptop to Home Network via OpenVPN on Cisco E3000 w/ Tomato USB firmware

This is a log to try OpenVPN.

——————

Environment:

Router: Cisco E3000 w/ Tomato 1.28-116 K26 by Shibby

Phone: iOS 7

Computer: Windows 8.1 x64

——————-

1. Download OpenVPN at: http://openvpn.net/index.php/open-source/downloads.html

choose Windows Installer (64bit), then install with all the components.

2. Creating the Certificates and keys

Run Windows Command Prompt (cmd) as Administrator:

cd C:Program FilesOpenVPNeasy-rsa

init-config

This will copy vars.bat and openssl.cnf to easy-rsa folder

notepad vars.bat

Open notepad and edit vars.bat, fill the following at the end of file:

set KEY_COUNTRY=
set KEY_PROVINCE=
set KEY_CITY=
set KEY_ORG=
set KEY_EMAIL=

Save. Then type the following: (when at the 3rd command => build-ca, hit ‘Enter’ for everything except ‘Common Name Parameter’, enter something, eg:  MyName) Continue reading Connect iPhone/Laptop to Home Network via OpenVPN on Cisco E3000 w/ Tomato USB firmware

Code::Blocks installation in CentOS 6.5 x64_minimal

A log of installing Code::Blocks in CentOS 6.5 x64 Minimal edition. (It’s a server, and codeblocks will be accessed on clients via putty and Xming.)

#Install some necessary libraries
yum groupinstall "Development Tools"  

#install rpmforge, there are other ways, here I use wget to download rpm package and install
#install wget
yum install wget
mkdir ~/temp
cd ~/temp
wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

rpm -ivh rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
yum install --enablerepo=rpmforge-extras

#Install wxGTK
yum install wxGTK-devel

#Install codeblocks from source
#ref:http://wiki.codeblocks.org/index.php?title=Installing_Code::Blocks_from_source_on_RPM_based_distributions
cd ~
mkdir codeblocks
cd codeblocks
svn checkout svn://svn.berlios.de/codeblocks/trunk
cd trunk
./bootstrap
./configure
make
sudo make install

#Now Code::Blocks has installed, configure remote access
#enable X11 forwarding: ref:http://www.techotopia.com/index.php/Displaying_CentOS_Applications_Remotely_(X11_Forwarding)
sudo vi /etc/ssh/ssh_config
#Edit the file and make sure -> X11Forward yes
sudo yum xorg-x11-xauth.x86_64
sudo yum install xorg-x11-fonts

#Now on a Windows client, using Putty and Xming access the server
#On windows desktop, download and install Xming Server and Xming fonts
#in Putty, enable X11 Forwarding, set display to "Localhost:0"

#Type the following:
echo $DISPLAY
#It should return localhost:10.0

#Run Code::Blocks
codeblocks &

 

Google Earth坐标趣味地址

这是我网上看到的一篇日志,出处不详, 但十分有意思有价值,全贴转载到本站了:

以下是转载:

————————————————–

“使用谷歌地球可以看见中国这片神奇的土地上一些被人们遗忘了的奇迹,比如在新疆戈壁滩上的“为人民服务”,每个字都有约2500平方米大,这几个大字不知什么时候就开始躺在了戈壁深处”。从标语的内容看有些应是60年代写的,字如此的清晰完整不知当时使用什么材料写的。除了“为人民服务”之外,还有以下这些标语,中国地域之广大,遍布许多地方,可能什么时间弄的也都有,找到这些标语也很有趣。现在,找到这些标语的方法很简单,每一处我都提供了坐标,只要启动“谷歌地球”(Google Earth),然后直接在Google的搜索栏中输入或粘贴坐标值后点搜索按钮,即可直接定位。(绝对不是PS,送坐标给你,自己去下个谷歌地球看看:直接在Google中输入坐标值后点Search 搜索按钮,即可直接定位。如有的地方不知坐标可直接输入名字点搜索按钮即可。

收集的一些Google Earth坐标趣味地址

大字标语

42 39 18.85 n,94 10 00.80 e   毛主席万万岁
42 32 33.95 n,94 19 36.80 e   为人民服务
42 27 12.08 n,94 08 49.36 e   排除万难去争取胜利
42 39 33.30 n,94 16 00.56 e   只争朝夕
42 27 40.95 n,94 14 36.80 e   向斗争中学习
39 41 43.75,73 55 37.39   祖国在我心中
27 51 29.88 N,99 43 13.73 E   香格里拉
27°52’5.10″N, 99°41’52.20″E 香格里拉-松赞林寺
45° 6 56.73 N, 82°31 32.44 E   严格要求严格训练
45°22 3.47 N, 90°34 23.67 E   卫国戍边以卡为家
43 45 16.93 N 87 40 03.75 E   仗怎么打,兵怎么练
99.69783333E,27.86808444N   香格里拉-松赞林寺
32°07’29.28″N, 114°00’08.64″E   一定要把淮河修好
41°49’13.62″N, 86°12’05.83″E    许身电业平生志 衣带渐宽终不悔
45°49’03.76″N 82°25’49.55″E     爱我中华!
45°22’2.53″N, 90°34’24.24″E      北塔山边防站:
24°27’21.00″N 118°10’10.40″E  厦门滨海大道上的,面对台湾金门的 宣传标语。

31°6’17.97″N  121°0’50.44″E    上海东方绿洲里的航母模型 Continue reading Google Earth坐标趣味地址