I always knew one way of resetting it, but recently came up with another which is The Right Way.

It works both for linux and win32 OSes, in my example it is linux. It requires just one MySQL restart and is more secure.

How to reset MySQL root password - The Right Way

Create text file with following SQL statements

GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY '[NEW_PASS]' WIth GRANT OPTION;

Replace [NEW_PASS] with desired new password and save this file somewhere like /tmp/pwd.sql

Update config my.cnf

Usually it is located in /etc/ or /etc/mysql/. Find section called [mysqld] and add following line

[mysqld]
init-file = /tmp/pwd.sql

Restart MySQL. Voila. Your new password should be in place.

Do not forget remove /tmp/pwd.sql and comment "init-file" option in config file, because without init-file found MySQL won't start next at all.

Ubuntu users warning!  In Ubuntu, MySQL is protected with Apparmor and is prohibited to read files from /tmp. In Ubuntu name file /etc/mysql/pwd.cnf, this way it is read by server process.

The other way - The Bad One

For some reason, the following method seems to be far more popular, starting MySQL with "--skip-grant-tables".

When MySQL is started with this parameter, it completely avoids checking its grant tables upon connection and upon query. This means anyone can log in from anywhere, and do anything on the database.

While the manual does mention this is a less preferred way of doing it, it does not elaborate. Starting MySQL with this parameter is a huge security breach. This is why one may wish to add the --skip-networking parameter, to only allow connection from the localhost (using Unix socket, for example).

Moreover, after MySQL starts, and the necessary GRANT or CHANGE PASSWORD take place, the server is still unsuitable for connections. This is why it needs to be restarted again, this time without --skip-grant-tables.

At first I quickly restored Apache, did all needed changes and switched back to Nginx. For sure, I did not like this way and explored how I can connect Mercurial directly to Nginx. I was quite surprised how easy it is and how flexible.

Continue reading...

Mercurial via Apache setup

I had Mercurial setup through Apache in a very standard way.

/var/hg - Holding all my repositaries

/var/hg/hgweb.config

[collections]
repos/ = repos/

/var/hg/hgwebdir.cgi - For Apache CGI support

/var/hg/hguser - for user passwords in htpasswd format

Apache configuration was the following

ScriptAliasMatch ^/hg(.*) /var/hg/hgwebdir.cgi$1
<Directory /var/hg>
  Options ExecCGI FollowSymLinks
  AllowOverride None
</Directory>

<Location /hg>
   AuthType Basic
   AuthName "Mercurial repositories"
   AuthUserFile /var/hg/hgusers
   Require valid-user
</Location>

Switch to Nginx was really painless.

Nginx and Mercurial setup

Prepare Mercurial

First step I switched to FastCGI instead of CGI. To do this I copied file hgwebdir.fcgi to my repo root /var/hg and made it executable

copy /usr/share/doc/mercurial-common/examples/hgwebdir.fcgi /var/hg
chmod +x /var/hg/hgwebdir.fcgi

Or grab the file here

wget http://selenic.com/repo/hg/raw-file/tip/contrib/hgwebdir.fcgi

Next step is to install something that can spawn FastCGI processes, sine Nginx can not do this by itself.  This is easy with spawn-cgi

#apt-get install spawn-fcgi

I configured it via local unix socket instead of TCP port, it seems to me this way it is more secure. Run it like this:

#spawn-fcgi -u www-data -g www-data -s /tmp/mercurial.sock -d /var/hg/ -- /var/hg/hgwebdir.fcgi

At first start I've run into the problem

spawn-fcgi: child exited with: 1

After running with "-n" switch to see debug output, I  realised that I was missing one python module needed for a setup.

Traceback (most recent call last):
 File "/var/hg/hgwebdir.fcgi", line 25, in
 from flup.server.fcgi import WSGIServer
 File "/usr/lib/pymodules/python2.6/mercurial/demandimport.py", line 106, in _demandimport
 mod = _origimport(name, globals, locals)
 ImportError: No module named flup.server.fcgi

To fix this I installed missing python packages and issue was fixed.

#apt-get install python-flup

Configure Nginx

Edit file /etc/nginx/sites-available/mercurial.conf and add the following server definition:

server {
   listen 80;
   server_name YOUR_MERCURIAL_DOMAIN;

   location / {
     auth_basic "Secure Login";
     auth_basic_user_file /var/hg/hgusers;
     include fastcgi_params;
     fastcgi_param PATH_INFO $uri;
     fastcgi_param REMOTE_USER $remote_user;
     fastcgi_intercept_errors on;
     fastcgi_pass unix:/tmp/mercurial.sock;
  } 
}

The above settings will setup a new virtual host, where all traffic is redirected to the Mercurial FastCGI wrapper. It is important that you forward the PATH_INFO and REMOTE_USER variables. Mercurial will not work correctly without these.

Be sure that file /var/hg/hgusers contains all your Mercurial users with passwords, it can be created using htpasswd tool shipped with Apache.

Restart Nginx and try to to push pull.

Misc

One thing that changed was repositary path. Before it has /hg/ in path line

http://a32.me/hg/project1

Now it become a full virtual host and changed to:

http://hg.a32.me/project1

I have to reconfigure all my projects, but it is not a pain like it was with CVS.

Next thing I though about, I can move repositories under its own uid:gid and not run it under www-data, this can improve security.

Do not forget to include running spawn-fcgi into /etc/rc.local for automatically start on system boot.

Links

My story is based on these links.

• http://www.opennet.ru/base/dev/mercurial_http.txt.html
• http://geeksharp.com/2010/01/20/mercurial-web-with-fastcgi-nginx/
• http://fosswire.com/post/2009/08/hosting-mercurial-repositories-with-nginx/
• http://www.softwareprojects.com/resources/programming/t-how-to-serve-mercurial-hg-repository-over-nginx-1856.html
• http://www.dikant.de/2009/07/29/running-mercurial-with-fastcgi-in-nginx/

Time changed things, community has grown, all sharp corners were smoothed. The PHP itself starting from version 5.3 has out-of-the-box support of FPM (FastCGI Process Manager) to communicate to Nginx in a very efficient manner. Ubuntu PPA repositories has all things already compiled and tons of docs are available on the Internet.

At first I though of this more like experiment, but after I tested it for a while and made some performance tests, I was surprised how well it performed on my tiny virtual server. It eats less memory and does more job.

I really like Apache web server, I've spend 10 years with it and still consider it the best one for many appliances. But from now on I have one more option to use for my projects.

Welcome to continue reading...

Actually there was one more reason to try Nginx - memory usage. Some time ago I moved my blog from VPS based on Virtuozzo to the Rackspace cloud based on XEN. This move was more related mostly to money savings. Same characteristic server  (10Gb HDD, 256MB RAM) is twice cheaper on the cloud, less lagging and more productive.

But this move revealed that XEN and Virtuozzo virtual machines counts free memory in different ways. I experienced memory shortage on XEN immediately after setup of LAMP stack and WordPress. Default Apache server limits are too optimistic and RAM is quickly filled up with Apache instances of 30Mb in size each. After lowering limits to values 2-3 of server instances situation got better.

All this time I knew it could be even better Once I found some time to do it, I've done it.

Nginx + php-fpm + WordPress

Setup is very easy, there are tons of blog posts, howtos and official documentation on the Internet. I chose the easiest one and following it have it up and running in half-hour.

I did all tests of my LIVE server with Apache running and did not want to crash my current blog activity. So I installed Nginx in parallel on another port (8080) and it went well. After successful tests I just swapped Apache and Nginx

Difference from Apache

In Apache, PHP usually works as module - mod_php, it is loaded into Apache address space and acts like part of it. All requests are handled by Apache instance, either it is static file it is sent directly from HD, in case of .php file, it is passed through mod_php and output is send to the client. Web requests are distributed through all Apache workers in round robin. Since mod_php eats lots of memory, all instances very soon become FAT.

Nginx has different approach. By default it serves only static files and does this in a very efficient way. Web requests to dynamic pages, to .php files, are redirected to special FPM daemon which runs this request though one of php-worker process and sends output back to Nginx and further back to the client. FPM is simular to CGI, but more efficient because can handle multiple requests with single process.

This difference means, small static files are served very quickly with small memory footprint and dynamic .PHP files are served by several FPM workers.

Installation on Ubuntu 10.04-LTS

Later versions of Ubuntu already have php5-fpm compiled, so no additional repositories are required.

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:nginx/stable
sudo add-apt-repository ppa:nginx/php5
sudo apt-get update
sudo apt-get install nginx php5-fpm

Configure php-fpm

It is a daemon which listens on localhost:9000 or unix socket for PHP requests, processes them balancing through available FPM workers. It also supports creating several pools of workers which may be running under different uid/gid or have different php.ini settings, this advanced usage is not covered here.

By default it is configured to have one default worker pool called www and if you are not concerned about performance or memory usage you may leave configuration as default.

I did some changes to limit number of processes to fit my little memory. Mine change to /etc/php5/fpm/pool.d/www.conf are:

listen=/tmp/php-fpm.sock
pm = dynamic
pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 2
pm.max_spare_servers = 2
pm.max_requests = 500

Configure Nginx

How to reach php-fpm. Unix socket seems better option for a local machine. I have this setting in separate config.

/etc/nginx/conf.d/php-fpm.conf

upstream php {
   server unix:/tmp/php-fpm.sock;
   #server 127.0.0.1:9000;
}

Main site config. I compiled it using several articles found and official one from WordPress.

/etc/nginx/sites-enabled/a32.me

server {
	listen	80;
        server_name a32.me;
        server_name www.a32.me;
        root /home/ameoba32/www/;

        index index.php;

	if ($http_host != "a32.me") {
	    rewrite ^ http://a32.me$request_uri permanent;
	}

        location = /favicon.ico {
                log_not_found off;
                access_log off;
        }

        location = /robots.txt {
                allow all;
                log_not_found off;
                access_log off;
        }

	# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).
	location ~ /\. {
	    deny all;
	    access_log off;
	    log_not_found off;
	}

	location / {
	    # No php is touched for static content
            try_files $uri $uri/ /index.php?$args;
        }

	# Add trailing slash to */wp-admin requests.
	rewrite /wp-admin$ $scheme://$host$uri/ permanent;

        location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
                expires max;
                log_not_found off;
        }

        location ~ \.php$ {
                expires off;
                try_files $uri =404;
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini

                include fastcgi_params;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_pass php;
        }
}

Launch

/etc/init.d/apache stop
/etc/init.d/php5-fpm start
/etc/init.d/nginx start

Update startup

iIf everything is well, do not forget to switch daemons on system start.

# update-rc.d apache remove
# update-rc.d nginx enable
# update-rc.d php5-fpm enable

What is the difference

OK, I moved, what really has changed ? I did several load tests using ApacheBench (ab) tool, which can generate load on a web server.

"-i" switch means to do HEAD requests instead of GET, difference is not to do actual content downloading, just  test pure server performance.

Test Apache, after stop and run Nginx and do several tests in a row to get average result.

Test of dynamic content, WordPress main page

$ ab -i -n 500 -c 20 http://a32.me/

Test of static content

$ ab -i -n 500 -c 20 http://a32.me/sitemap.xml

Apache vs Nginx on dynamic PHP pages

Apache PHP serving

Requests per second: 6.81 [#/sec] (mean)
Time per request: 2935.476 [ms] (mean)
Time per request: 146.774 [ms] (mean, across all concurrent requests)

Nginx PHP serving

Requests per second: 14.27 [#/sec] (mean)
Time per request: 1401.530 [ms] (mean)
Time per request: 70.076 [ms] (mean, across all concurrent requests)

Apache vs Nginx on static content

Apache static content serving

Requests per second: 15.38 [#/sec] (mean)
Time per request: 1300.093 [ms] (mean)
Time per request: 65.005 [ms] (mean, across all concurrent requests)

Nginx static content serving

Requests per second: 55.84 [#/sec] (mean)
Time per request: 358.182 [ms] (mean)
Time per request: 17.909 [ms] (mean, across all concurrent requests)

Apache vs Nginx conclusion

On dynamic files I've got it 109% increase in efficiency, on static content 263%. All this by simple changing the web server software. I am very satisfied with simplicity and performance it gives. Nginx setups are also easy to scale using several servers.

Nginx mod_rewrite

One drawback I've found after move is that .htaccess files are not supported in the way Apache does. To overcome this .htaccess configuration has to be converted into Nginx comliant configuration and inserted into directly into Nginx main config.

I've found nice converter that can help to do this

http://www.anilcetin.com/convert-apache-htaccess-to-nginx/

Links

• http://codex.wordpress.org/Nginx
• http://wordpress.org/extend/plugins/nginx-compatibility/
• http://www.thelinuxgeeks.info/howto-install-nginx-php-cgi-on-ubuntu/

Being a web developer this topic is in area of my interests for some time, but I always thought that clouds are for really big players. By big I assume million of clients, billions of page loads, terabytes of data, etc.. My mistake!

Recently I came across with Rackspace cloud and understood how wrong I was. It changed my understanding  into more practical way and I gave up my VPS and moved to the cloud.

What is the cloud

Cloud is something where I can put all my data and my code and it will work somehow, by definition :) In real life cloud is a hosting, thats it. It looks and acts like a VPS hosting.

The major differences are described below:

Quick spin up

You may remeber your first purchase of  VPS hosting service. It took about 24 hours to get your new instance working. In cloud it is a little different. All you need is to enter your admin panel and order new server. You may choose CPU, RAM, HD space and operating system (10+ Linux distribution or windows OS). In minutes your server is ready to use and you may login with SSH and start your business.

Either you need new server for new project or just for testing purposes or your load increased, it is easy to create and delete servers in a minutes.

Scale to meed demands

Need a bigger server? Hit the upgrade button and re-size your server instantly with more RAM, CPU and space. Useful for increased load, there is no work for migration.

Pay as you go

In case creating and deleting servers is an easy task it also means it is impossible to take money per month per sever. In the cloud there is a different approach: pay per second. You pay only for time your server is UP and running. In case new server is needed just for couple of days, you will be charged for that couple of days.

API

All cloud providers has Application Programmable Interface (API). This means simple thing, all actions which can be made though admin panel can be made by automatically by program. This adds great scalability for application. Here is the case: your  project is very successful and is aimed to North America continent customers. Usually during a day it has activity peak for 8 hours, but for rest 16 hours it is almost idle. Using Cloud API it is simple to spin up additional servers by cron task to handle peak load and kill them after. This saves money.

Load balancer

Load balancer is a simple box which balances or distributes traffic directed to one IP address to many server.

Each new connection is redirected to another server, this way it spreads site load on many server. Once one server is not enough to handle all user visits and page views it is easy to add more and more servers behind load balancer. One can be setup through admin panel in a minutes.

CDN

Content Delivery Network - some sort of storage which can survive with high traffic content. It is up to cloud provider how they do it. Your duty is to upload your high traffic content like site images, user pictures, downloadable files to some cloud area and this content is always available  at high speed to all world locations.

My experience

I have rented VPS for a long time, it was cheap one, slow one. About a month ago I discovered Rackspace Cloud server, which conquered me with its speed, support and most important the price. Price entry to Rackspace Cloud is low comparing to other services.

Cheapest server with 256MB of RAM, 10GB of space is just 1.5¢/per hour that is about 11$/mth with great performance.

Highly recommended

cloud tech move to the cloud whats the cloud

A must have thing for every admin. If you are Debian or Ubuntu admin it still worth reading, since principles stays the same and there is no much real difference in security between RHEL and Ubuntu.

Link to RHEL5 secure guide NSA site (local copy)

Quick one A4 page pamphlet (local copy)

Besides RHEL, there are many guides for other OSes as well

• Secure configuration of Mac OS X 10.6 Snow Leopard (October 2010)
• Secure configuration of Mac OS X 10.5 Leopard (March 2009)
• Using of Software Restriction Policies (SRP) in Windows (March 2011)
• Security highlights in Windows 7 (May 2011)
• Windows Vista security guide (March 2009)

Additional:

Debian and Ubuntu security guide (local copy)

.

Put this script into cron to run at each hour and you will get history for your connection.

#!/bin/sh

dir=/var/log/ping/`date +%Y.%m.%d`
fn=`date +%H.00`

mkdir -p $dir

# Ping gateway and google dns for 1 hour
ping -c 3601 -i 1 [GW_IP] > $dir/$fn.gw&
ping -c 3601 -i 1 8.8.8.8 > $dir/$fn.gog&

# Wait for pings to finish
wait

# Gzip final files
gzip $dir/$fn.gw
gzip $dir/$fn.gog

# Clean up old files for 3 month
find /var/log/ping/ -type f -mtime +90 -exec rm {} \;