PHP (Symfony) Profiling with XHProf and XHGui
#which pecl
#which pear
#port search –name –line –glob 'php*' |grep -i pear
php5-pear 20161027 php www Obsolete port, replaced by php55-pear
php53-pear 20161027 php www Optional port adds the PEAR repository to the include path for php53.
php54-pear 20161027 php www Optional port adds the PEAR repository to the include path for php54.
php55-pear 20161027 php www Optional port adds the PEAR repository to the include path for php55.
php56-pear 20161027 php www Optional port adds the PEAR repository to the include path for php56.
php70-pear 20161027 php www Optional port adds the PEAR repository to the include path for php70.#sudo port install php55-pear
No trace of PECL on the system yet so try another PEAR installation:
#sudo curl -o /usr/local/bin/pear http://pear.php.net/go-pear.phar
#sudo chmod +x /usr/local/bin/pear
#which pecl
It seems that PECL did not install with the methods above so take the third approach:
#sudo php /usr/lib/php/install-pear-nozlib.phar -d /usr/local/lib/php -b /usr/local/bin
#which pecl
/usr/local/bin/pecl
2. Install the modules for PHP starting with XHProf:
#php -v
PHP 5.5.38 (cli)
#port search –name –line –glob 'php*' |grep -i xh
php-xhprof 0.9.4 php devel A Hierarchical Profiler for PHP
php53-xhprof 0.9.4 php devel A Hierarchical Profiler for PHP
php54-xhprof 0.9.4 php devel A Hierarchical Profiler for PHP
php55-xhprof 0.9.4 php devel A Hierarchical Profiler for PHP
php56-xhprof 0.9.4 php devel A Hierarchical Profiler for PHP
#sudo port install php55-xhprof
3. Before installing the XHGui module for php, install its requirements: MongoDB and its PHP modules:
#port search –name –line –glob 'php*' |grep -i mongo
php-mongo 1.6.14 php databases devel Mongo Database Driver
php-mongodb 1.1.9 php databases devel MongoDB Database Driver
php53-mongo 1.6.14 php databases devel Mongo Database Driver
php53-mongodb 1.0.1 php databases devel MongoDB Database Driver
php54-mongo 1.6.14 php databases devel Mongo Database Driver
php54-mongodb 1.1.9 php databases devel MongoDB Database Driver
php55-mongo 1.6.14 php databases devel Mongo Database Driver
php55-mongodb 1.1.9 php databases devel MongoDB Database Driver
php56-mongo 1.6.14 php databases devel Mongo Database Driver
php56-mongodb 1.1.9 php databases devel MongoDB Database Driver
php70-mongodb 1.1.9 php databases devel MongoDB Database Driver#sudo port install php55-mongodb
#sudo port install php55-mongo
#ls /opt/local/lib/php55/extensions/no-debug-non-zts-20121212/
mongo.so mongodb.soOn my system the modules were placed in another folder than the one where PHP is configured so a copy is needed:
#sudo cp /opt/local/lib/php55/extensions/no-debug-non-zts-20121212/mongo* /usr/lib/php/extensions/no-debug-non-zts-20121212/
#sudo cp /opt/local/lib/php55/extensions/no-debug-non-zts-20121212/xhprof.so /usr/lib/php/extensions/no-debug-non-zts-20121212/
Continue by enabling the extensions:
#sudo vi /etc/php.ini
extension=mongo.so
extension=mongodb.so
extension=xhprof.so
This command actually installs the MongoDB database. The ones above installed the php driver only :
#brew install mongodb
#sudo chgrp staff /usr/local/var/log/mongodb /usr/local/var/mongodb
#sudo chmod 0775 /usr/local/var/log/mongodb /usr/local/var/mongodb
Add this to /usr/local/etc/mongod.conf:
processManagement:
fork: true#mongod –config /usr/local/etc/mongod.conf
Optional, use a tool to install Pref into System Preferences in order to start mongodb
#cd <downloads>
#wget https://github.com/remysaissy/mongodb-macosx-prefspane/raw/master/download/MongoDB.prefPane.zip
#unzip MongoDB.prefPane.zip
double-click on MongoDB.prefPane to install it and launch it
4. Install XHGui and configure it as a Virtual Server:
#cd <some directory for xhgui>
#git clone https://github.com/perftools/xhgui.git
#cd xhgui/
#chmod 0777 cache/
#php install.php#sudo vi /etc/apache2/extra/httpd-vhosts.conf
<VirtualHost *:80>
ServerAdmin [email protected]
DocumentRoot "/Sites/xhgui/webroot"
ServerName xhgui.lh
ServerAlias www.xhgui.lh
ErrorLog "/private/var/log/apache2/xhgui.lh-error_log"
CustomLog "/private/var/log/apache2/xhgui.lh-access_log" common
</VirtualHost>#sudo vi /etc/hosts
127.0.0.1 xhgui.lh www.xhgui.lh#sudo apachectl restart
Optional, but recommended: Add indexes to MongoDB to improve performance. (https://github.com/perftools/xhgui)
# mongo
use xhprof
db.results.ensureIndex( { 'meta.SERVER.REQUEST_TIME' : -1 } )
db.results.ensureIndex( { 'profile.main().wt' : -1 } )
db.results.ensureIndex( { 'profile.main().mu' : -1 } )
db.results.ensureIndex( { 'profile.main().cpu' : -1 } )
db.results.ensureIndex( { 'meta.url' : 1 } )
exit
5. Add this to VirtualHost of the site needing profiling (/etc/apache2/extra/httpd-vhosts.conf):
php_admin_value auto_prepend_file "/Sites/xhgui/external/header.php"
That's it. See it in browser:
http://www.xhgui.lh/
Mercurial is a cross-platform, distributed revision control tool for software developers. In this article will be presented a handy way to install and configure Mercurial on a server with cPanel . All operations presented will only need a standard FTP/SSH access to a user account, except the initial installation of the Mercurial RPM which needs root rights. This installation procedure is very suitable for those creating websites and wanting a quick solution to track the changes in their source code using a repository stored right next to the website.
The information below were compiled from several sources with some modifications to suit the above needs. The aim is to use only cPanel and SSH with user rights (e.g. no root required !) and have a repository ready in 5 minutes. The repository will then be accessed securely as hg.yoursite.com
Intial preparation: install the Mercurial package as root http://mercurial.selenic.com/wiki/HgWebDirStepByStep
Here is a sample installation from a RPM on an OpenSuse linux:
wget http://download.opensuse.org/repositories/devel:/tools:/scm/openSUSE_10.2/x86_64/mercurial-1.0-1.2.x86_64.rpm
$ sudo rpm -ihv mercurial-1.0-1.2.x86_64.rpm
Step 1. Create subdomain
Login in cPanel and create hg.yourdomain.com. Do not create the directory under /public_html so the repository won't mix with the files of the website
Step 2. Create the configuration file of the repository
Use a FTP client and navigate into the directory of the subdomain. Create a folder for your repo, i.e. projectrepo . Use some name other than the one you want to access the repo with because later there will be some URL rewiting rules added into a .htaccess file which won't work otherwise. In this example the repo url will be made as hg.yourdomain.com/project
Continue navigating into cgi-bin where two files has to be be uploaded: hgweb.config containing the repository configuration and hgwebdir.cgi containing the code needed to run the repository. Create the two files on your computer and upload them into /cgi-bin:
/cgi-bin/hgweb.config
[paths]
#VIRTUAL_PATH = /REAL/PATH
project = ~/hg/projectrepo/cgi-bin/hgwebdir.cgi
#!/usr/bin/env python
from mercurial import demandimport; demandimport.enable()import cgitb
cgitb.enable()
import os
os.environ["HGENCODING"] = "UTF-8"
from mercurial.hgweb.hgwebdir_mod import hgwebdir
import mercurial.hgweb.wsgicgi as wsgicgi
application = hgwebdir('hgweb.config')
wsgicgi.launch(application)
Change the attributes of hgwebdir.cgi to be executable (0755). This can be made within the FTP client (at least TotalCommander implements it and probably many other such applications have that option. It is under Files->Change attributes).
Step 3. Initialize repository
Using a SSH console go into the ~/hg folder and run these commands:
hg init
find . -type f -exec chmod 0666 "{}" \;
find . -type d -exec chmod 0777 "{}" \;
The last two commands are needed when the web server (Apache) runs the scripts under a different user than the owner of the account. In order to create files and folder full write rights are needed for user apache or nobody.
A configuration file will have to be created and uploaded into the new repository. For this , create a file named hgrc with the following content:
~/hg/project/.hg/hgrc
[web]
description = My repository
push_ssl = false
allow_push = john# enable snapshot downloads
allow_archive = gz zip bz2
baseurl = /
In this configuration the ssl is disabled which may be a security problem. If your host allows https connections then you may set this to true later. However, the repository is protected through HTTP Basic authentication – see below.
Upload the file above under ~/hg/project/.hg/
Step 4. Secure the repository
Create a .htaccess file with the following content for uploading in the root of your subdomain:
~/hg/.htaccess
# Taken from http://www.pmwiki.org/wiki/Cookbook/CleanUrls#samedir
# Used at http://ggap.sf.net/hg/
Options +ExecCGI
RewriteEngine On
#write base depending on where the base url lives
RewriteBase /cgi-bin
RewriteRule ^$ hgwebdir.cgi [L]
# Send requests for files that exist to those files.
RewriteCond %{REQUEST_FILENAME} !-f
# Send requests for directories that exist to those directories.
RewriteCond %{REQUEST_FILENAME} !-d
# Send requests to hgwebdir.cgi, appending the rest of url.
RewriteRule (.*) /cgi-bin/hgwebdir.cgi/$1 [QSA,L]
The rules above allows accessing the repository with an url like hg.yourdomain.com/project instead of hg.yourdomain.com/cgi-bin/hgwebdir.cgi/project and is a replacement for the original procedure described in the Mercurial documentation where one has to modify httpd.conf, as root.
Finally, password protect the repository within cpanel. The operations below will update .htaccess, adding the necessary lines for basic authentication:
Finish
Access your repository within a browser with an URL like:
http://john:[email protected]/hg/project
Use the URL above in your Mercurial client too.
Referrences
The authentication methods are a great tool for fighting against spammers and every server admin should have it implemented on their systems. But since it is not that simple to configure them, they often prove to be a headache with not enough documentation available. I had to pay myself the tribute (in hours spent) for making the system work in the configuration I wanted and for the others having the same problems here is my story.My configuration is : CentOS 5, Postfix 2.3.3
The server is running some websites and I wanted to be capable of sending signed emails within PHP and also using some email client through SMTP. Since there are more than one website I wanted also a simple way to add/modify domains in the filters' configuration.
Whenever I write "domain.tld" you should use your own domain name (mysite.com, wikitiki.org, etc)
The tools I installed are listed here under the Authentication section. They are:
Installing dkim. For this follow one of these links:
How to Setup DomainKeys Identified Mail (DKIM) with Postfix and Ubuntu Server
Setting up DKIM, SPF, Domainkeys DNS, Regular DNS on CentOS 5.3
I won't go through the installation steps you can read about in tutorials as above. Instead I will focus on what should be checked, customized, verified so that will help in custom installations or troubleshooting.
So, for the installation I will focus on:
| subject | value | description |
|---|---|---|
| username/usergroup | domainkeys.domainkeys | This is the owner of the signer/verifier processes. Doesn't matter the names but once created will be used for all filters |
| folders for configuration | /etc/mail/dkim/ /etc/mail/domainkeys/ |
These must be owned by the user above. When adding new domains I have to modify some files here |
| initialization scripts | /etc/init.d/dkim /etc/init.d/domainkeys /etc/init.d/sender-id |
Essentially is the same start-up script configured for each filter. Good to know where they are when testing the configuration |
| folders for private keys | /var/db/domainkeys/domain.tld/ | These must be owned by the user above. Here I'll place the private keys |
| folders & files for processes' PIDs | /var/run/dkim-milter/pid /var/run/dk-milter/pid /var/run/dk-milter/pid |
These also have to be owned by the user above. The filters will create/remove the "pid" files as they need |
OK, so here there are some background information. DKIM and Domainkeys uses two DNS records and a private key to sign the emails. The DNS records are
_domainkey.domain.tld IN TXT "t=y;o=~"
o=~ means that some emails will be signed. "o=-" means that all domains are signed
default._domainkey.domain.tld IN TXT "g=*; k=rsa; p=MIGfMA0...ABC"
"_domainkey" is a keyword preceding the domain name
"k=" key type. Optional. Default is rsa
"p=" public key data.
"g=" granularity of the key. Optional, default is "*"
"v=DKIM1" version. It is recommended but I got some errors when I tried to use the same record for Domainkeys because it is defined for DKIM only so I did not use it. Here is the yahoo tool I tested with
Note: in the two dns records above I used "domain.tld" only because my external nameserver service required it in its web interface. If you own the namesever you will probably not use it as will be appended automatically by named/bind
Important : the server may sign ALL emails with its own domain name or may sign it with other domains but in this case all other domains must have the two DNS records maintained. I went for the first option.
there are three ways to generate them:
openssl genrsa -out private.key 1024 openssl rsa -in private.key -out public.key -pubout -outform PEM
dkim-genkey -r -d domain.tld
The result will always be two files, one private key to copy into /var/db/domainkeys/domain.tld/default and one to use it in the DNS record as value for "p=" parameter. Pay attention that I used "default" as filename for the private key in order to use that file for a selector named "default" and defined as such in the DNS record. It is possible to have different selectors which would handle different emails or domains and so I can have:
/var/db/domainkeys/domain.tld/m1.pem – this is created for the selectod m1 and I added an extension just to remember it is a private key file. the extension, if exists, must be "pem" or "private".
/var/db/domainkeys/otherdomain.tld/default – this one will serve emails for another domain with another selector named "default"
The init script will start|stop the filter and there is where I put the command to do that. It may have two forms, either with the configuration given as parameters to the executable app or with it uses a single parameter which points to a configuration file containing all settings. I used the 2nd option:
COMMAND="dkim-filter -x /etc/dkim.conf"
KeyList /etc/mail/dkim/keylist
MTA MSA
Selector default
UserID domainkeys:domainkeys
Socket inet:8891@localhostDon't forget that the port numbers will be used in your /etc/postfix/main.cf file:
smtpd_milters = inet:localhost:9967,inet:localhost:8891,inet:localhost:8892
non_smtpd_milters = inet:localhost:9967,inet:localhost:8891,inet:localhost:8892
milter_protocol = 2
{!{code}!}czozMDpcIm1pbHRlcl9kZWZhdWx0X2FjdGlvbiA9IGFjY2VwdFwiO3tbJiomXX0={!{/code}!}the keylist file contains the relation between the sender-pattern, the signing domain and the path to the private key. I use this keylist file in order to manage easier the domains on my webhost server. For a singe domain one would only use the dkim's configuration file. My keylist file's location is defined in the configuration :
KeyList /etc/mail/dkim/keylist
And contains :
*:domain.tld:/var/db/domainkeys/domain.tld/default
The selector used in the signature will be the filename portion of keypath. If the file referenced by keypath cannot be opened, the filter will try again by appending ".pem" and then ".private" before giving up. (man dkim-filter.conf)
postsuper -d ALL
mail [email protected]
telnet localhost 25 EHLO domain.tld MAIL FROM: [email protected] RCPT TO: [email protected] data bla bla . quit
php -r "mail('[email protected]','test mail','bla bla','From: [email protected]');"A note here: At a certain point my mail sent from PHP was not signed with Domainkeys, only with DKIM. After several tests I found that the $headers in my php script were using Windows separator for new like (\r\n) instead of Linux (\n). I never thought it matters in headers !
tail -f /var/log/maillog
As above, start with the download & installation from the given links from sourceforge or with the ones below:
Quick And Easy Setup For DomainKeys Using Ubuntu, Postfix And Dkim-Filter
man dk-filter – DomainKeys filter for sendmail online version of the man page for dk-filter
The DNS records follows the same rules as the DKIM above. In fact both DKIM and Domainkeys will use the same DNS records
The dk-filter does not support all parameters supported by DKIM and especially the one which would allow me to put all configuration in one file therefore the command from this initialization script has all parameters in one line:
COMMAND="dk-filter -u $USER -p inet:$PORT@localhost -l -P $PIDFILE -d $DOMAIN -S $SELECTOR -c simple -k -s $KEYFILE -i $IFILE -m $SUBMISSION_DAEMON -D"
Parameters:
The content of the keylist (differs from DKIM's !):
*@domain.tld:/var/db/domainkeys/domain.tld/default
192.168.1.0/24127.0.0.1
This is another small difference than the DKIM's keylist file content. It looks like this (/etc/mail/domainkeys/keylist) :
*@domain.tld:/var/db/domainkeys/domain.tld/default
At some moment I got in maillog : "can't read SMFIC_EOH reply packet header: Success" which was solved after setting the SUBMISSION_DAEMON above as MSA (instead of smtp as I saw somewhere)
Playing with milter_protocol=4 in /etc/postfix/main.cf I got "can't read SMFIC_DATA reply packet header: Success" which was because I've given a value which was not supported. Also milter_protocol=6 thrown error so I let it set on : milter_protocol = 2
postconf -d
postconf -n
/usr/sbin/sendmail -t -f [email protected] [email protected] From: [email protected] Subject: test it bla bla .
ps aux |grep dk
This would be the 3rd filter installed and is aimed for hotmail accounts. There is a confusion between Sender-ID and SPF because they both use the same DNS registration record but the implementation of the Sender-ID protocol is similar with DKIM above. Read more about SPF vs Sender ID
Note that in order to sign the emails correctly one would only need the SPF record – see next section. At least this should work for 80% of the emails sent to hotmail/MSN according to the link above.
I named the init script "/etc/init.d/sender-id" and the command used to start the filter is:
COMMAND="sid-filter -u $USER -t -p inet:$PORT@localhost -l -P $PIDFILE -d $DOMAIN -D -h"
The parameters means:
The SPF authentication is used by google and is quite simple requiring no software installation on the server. In order to generate a SPF record for your domain you will probably have to go to the most-used tool :
Setup wizard to create SPF records for your domain
The record it will create should be placed as a TXT record at your domain and looks like this:
v=spf1 a a:external.domain.tld ~all
Where:
Definition: Mechanisms can be used to describe the set of hosts which are designated outbound mailers for the domain.
Observation: the SPF standard specify that there should be used the new DNS registration record type SPF instead of TXT but because of compatibility with the current DNS implementations it is recommended to put both TXT and SPF records with the same content.
Here are some of the tools and commands I used to verify that the record is in place and the configurations above are working:
dig domain.tld TXT
;; ANSWER SECTION: domain.tld. 300 IN TXT "v=spf1 a a:external.domain.tld ~all"
Thank you for using the verifier,cd /etc/init.d/; chkconfig --add sender-id
I invite any reader to help improving this newbie-guide based on their experience and knowledge. There is plenty room for suggestions and additions to this article but this is an area where with all documentation available it seems that it is not enough or not covers all configuration or problems which may occur when setting up such a system. I spent myself several (many) good hours reading and learning, trying and failing or succeeding and finally writing this article hoping that will help others to save some of their time and neurons while trying to follow the latest recommendation of the war against spammers.
Although not required, a donation is always welcome end encouraging 😉
|
|
| using PayPal |
DKIM configuration file /etc/dkim.conf
DKIM startup script /etc/init.d/dkim
Domainkeys startup script /etc/init.d/domainkeys
Sender-id startup script /etc/init.d/sender-id
How to Setup DomainKeys Identified Mail (DKIM) with Postfix and Ubuntu Server
http://www.unibia.com/unibianet/systems-networking/how-setup-domainkeys-identified-mail-dkim-postfix-and-ubuntu-server
Setting up DKIM, SPF, Domainkeys DNS, Regular DNS on CentOS 5.3 at Pacificrack.com
http://palma-seo.com/setting-dkim-spf-domainkeys-dns-bind
Set Up DKIM On Postfix With dkim-milter (CentOS 5.2)
http://www.howtoforge.com/set-up-dkim-on-postfix-with-dkim-milter-centos-5.2
DomainKey Implementor's Tools and Library for email servers & clients
http://domainkeys.sourceforge.net/
Set Up DKIM For Multiple Domains On Postfix With dkim-milter 2.8.x (CentOS 5.3)
http://www.howtoforge.com/set-up-dkim-for-multiple-domains-on-postfix-with-dkim-milter-2.8.x-centos-5.3
Setting Up SPF, SenderId, Domain Keys, and DKIM
Domain Key / DKIM Key Generation Wizard
http://www.socketlabs.com/services/dkwiz
Domainkeys and DKIM with Postfix – Gentoo Linux wiki
http://en.gentoo-wiki.com/wiki/Domainkeys_and_DKIM_with_Postfix
Postfix Add-on Software
http://www.postfix.org/addon.html
DomainKeys Identified Mail (DKIM) Signatures – RFC
http://www.ietf.org/rfc/rfc4871.txt
Postfix / DKIM
https://help.ubuntu.com/community/Postfix/DKIM
How To Implement SPF In Postfix
http://www.howtoforge.com/postfix_spf
Email authentication guide made by BITS
http://www.bits.org/downloads/Publications%20Page/BITSSenderAuthenticationDeploymentFINALJun09.pdf
Return Path’s Best Practice Guide
http://www.returnpath.net/downloads/resources/Email%20Authentication%20v080107.pdf
Last update: May 5th, 2010