any optimization suggestions :)

Blog Category: cfzr.com — Blogged by: mike on March 12, 2010 at 5:13 pm
  • Hi There, Eva and others :)

    My web server used to crash twice a day till i changed the ram from 512 to 2 gig, it is ok now, but it still slows down alot in peak time.

    I have two peak times a day, and i have no problem at all in other times, here is the required information about the server, please note that this is not a peak time stats.

    any suggestion to optimize the server config is appreciated :) ...

    and a quick question, generaly speaking: is it better to have one powerful server, or divid the load by two using the new feature in vb3.

    1- Dedicated (from 15minuteservers.com)

    2 - Pentium-4 2.4 Ghz, 2Gigs Ram, 80 GB IDE , Fedora-CR3
    Apache 1.3.33 , DirectAdmin 1.25.4 , Exim 4.50 , MySQL 4.1.10a
    Daily Pageviews 150,000 to 180,000
    Posts: 1.8 Million posts.
    Some features are disabled or hidden (For performance reasons):
    - Online users, emails, calendar, new posts, images, visit counts(delayed) , and most of the features that affects performance.


    3- No

    4- No Idea ..

    5- top - 19:16:24 up 56 days, 7:49, 1 user, load average: 0.21, 0.29, 0.27
    Tasks: 230 total, 1 running, 229 sleeping, 0 stopped, 0 zombie
    Cpu(s): 38.0% us, 3.6% sy, 0.3% ni, 54.8% id, 2.8% wa, 0.2% hi, 0.3% si
    Mem: 2074164k total, 1876656k used, 197508k free, 76612k buffers
    Swap: 2048276k total, 284900k used, 1763376k free, 1296592k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    7499 root 16 0 2632 1016 700 R 3.9 0.0 0:00.02 top
    6369 apache 15 0 29848 10m 2144 S 1.9 0.5 0:07.34 httpd
    7497 root 16 0 5712 1680 1372 S 1.9 0.1 0:00.01 sshd
    1 root 16 0 1840 496 472 S 0.0 0.0 0:51.63 init
    2 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
    3 root 34 19 0 0 0 S 0.0 0.0 0:00.43 ksoftirqd/0
    4 root 5 -10 0 0 0 S 0.0 0.0 2:48.20 events/0
    5 root 7 -10 0 0 0 S 0.0 0.0 0:00.01 khelper
    17 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
    100 root 5 -10 0 0 0 S 0.0 0.0 7:20.48 kblockd/0
    169 root 9 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
    168 root 15 0 0 0 0 S 0.0 0.0 158:30.29 kswapd0
    108 root 15 0 0 0 0 S 0.0 0.0 0:00.00 khubd
    260 root 15 0 0 0 0 S 0.0 0.0 0:00.00 kseriod
    467 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 ata/0
    480 root 25 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_0
    481 root 25 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_1
    489 root 15 0 0 0 0 S 0.0 0.0 1:05.00 kjournald
    1456 root 6 -10 3440 388 388 S 0.0 0.0 0:00.02 udevd
    2094 root 15 0 0 0 0 S 0.0 0.0 0:00.03 kjournald
    2095 root 15 0 0 0 0 S 0.0 0.0 0:46.51 kjournald
    2096 root 15 0 0 0 0 S 0.0 0.0 6:56.31 kjournald
    2097 root 15 0 0 0 0 S 0.0 0.0 0:09.37 kjournald
    2098 root 15 0 0 0 0 S 0.0 0.0 3:23.29 kjournald
    3063 root 16 0 3188 544 476 S 0.0 0.0 1:08.75 syslogd
    3067 root 16 0 2936 460 404 S 0.0 0.0 0:00.02 klogd
    3121 root 16 0 5708 860 836 S 0.0 0.0 0:02.38 rpc.idmapd
    3277 root 16 0 2912 556 480 S 0.0 0.0 3:22.11 nifd
    3293 root 20 0 2524 444 444 S 0.0 0.0 0:00.00 acpid
    3360 root 15 0 4612 1200 1088 S 0.0 0.1 2:09.33 sshd
    3392 mail 15 0 8104 1536 1444 S 0.0 0.1 0:25.22 exim
    3463 ftp 16 0 3396 736 624 S 0.0 0.0 0:10.88 proftpd
    3473 root 16 0 3248 436 436 S 0.0 0.0 0:00.00 vm-pop3d
    3492 root 16 0 4360 712 656 S 0.0 0.0 0:10.14 crond
    3545 daemon 16 0 2516 600 564 S 0.0 0.0 0:00.73 atd
    3561 dbus 16 0 3384 592 592 S 0.0 0.0 0:00.04 dbus-daemon-1
    3576 root 16 0 7696 1668 724 S 0.0 0.1 1:29.70 hald
    3627 root 18 0 2032 348 348 S 0.0 0.0 0:00.00 mingetty
    3630 root 19 0 2360 348 348 S 0.0 0.0 0:00.00 mingetty
    3631 root 18 0 2520 348 348 S 0.0 0.0 0:00.00 mingetty
    3632 root 18 0 3232 348 348 S 0.0 0.0 0:00.00 mingetty

    6- [mysqld]
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    skip-name-resolve
    skip-innodb
    max_connections = 650
    key_buffer = 32M
    myisam_sort_buffer_size = 64M
    join_buffer_size = 1M
    read_buffer_size = 1M
    sort_buffer_size = 2M
    table_cache = 1800
    thread_cache_size = 128
    wait_timeout = 900
    connect_timeout = 10
    tmp_table_size = 64M
    max_allowed_packet = 16M
    max_connect_errors = 10
    read_rnd_buffer_size = 524288
    bulk_insert_buffer_size = 8M
    query_cache_limit = 2M
    query_cache_size = 96M
    query_cache_type = 1
    query_prealloc_size = 16384
    query_alloc_block_size = 16384

    [mysql.server]
    user=mysql
    basedir=/var/lib

    [mysqld_safe]
    err-log=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid
    open_files_limit = 8192

    [mysqldump]
    quick
    max_allowed_packet = 16M

    [myisamchk]
    key_buffer = 64M
    sort_buffer = 64M
    read_buffer = 16M
    write_buffer = 16M


    7- +--------------------------+------------+
    Variable_name Value
    +--------------------------+------------+
    Aborted_clients 139670
    Aborted_connects 17189
    Binlog_cache_disk_use 0
    Binlog_cache_use 0
    Bytes_received 3592538822
    Bytes_sent 3512067873
    Com_admin_commands 4
    Com_alter_db 0
    Com_alter_table 4
    Com_analyze 0
    Com_backup_table 0
    Com_begin 0
    Com_change_db 7846314
    Com_change_master 0
    Com_check 0
    Com_checksum 0
    Com_commit 0
    Com_create_db 2
    Com_create_function 0
    Com_create_index 0
    Com_create_table 35
    Com_dealloc_sql 0
    Com_delete 820817
    Com_delete_multi 0
    Com_do 0
    Com_drop_db 2
    Com_drop_function 0
    Com_drop_index 0
    Com_drop_table 0
    Com_drop_user 0
    Com_execute_sql 0
    Com_flush 6
    Com_grant 4
    Com_ha_close 0
    Com_ha_open 0
    Com_ha_read 0
    Com_help 0
    Com_insert 5055745
    Com_insert_select 136068
    Com_kill 0
    Com_load 0
    Com_load_master_data 0
    Com_load_master_table 0
    Com_lock_tables 6
    Com_optimize 0
    Com_preload_keys 0
    Com_prepare_sql 0
    Com_purge 0
    Com_purge_before_date 0
    Com_rename_table 0
    Com_repair 0
    Com_replace 574635
    Com_replace_select 0
    Com_reset 0
    Com_restore_table 0
    Com_revoke 0
    Com_revoke_all 0
    Com_rollback 0
    Com_savepoint 0
    Com_select 36271217
    Com_set_option 401
    Com_show_binlog_events 0
    Com_show_binlogs 0
    Com_show_charsets 0
    Com_show_collations 0
    Com_show_column_types 0
    Com_show_create_db 0
    Com_show_create_table 394
    Com_show_databases 529
    Com_show_errors 0
    Com_show_fields 395
    Com_show_grants 0
    Com_show_innodb_status 0
    Com_show_keys 0
    Com_show_logs 0
    Com_show_master_status 0
    Com_show_new_master 0
    Com_show_open_tables 0
    Com_show_privileges 0
    Com_show_processlist 0
    Com_show_slave_hosts 0
    Com_show_slave_status 0
    Com_show_status 1
    Com_show_storage_engines 0
    Com_show_tables 766
    Com_show_variables 171
    Com_show_warnings 0
    Com_slave_start 0
    Com_slave_stop 0
    Com_truncate 0
    Com_unlock_tables 7
    Com_update 9574977
    Com_update_multi 0
    Connections 7891180
    Created_tmp_disk_tables 50436
    Created_tmp_files 19215
    Created_tmp_tables 407044
    Delayed_errors 0
    Delayed_insert_threads 0
    Delayed_writes 0
    Flush_commands 1
    Handler_commit 0
    Handler_delete 2172017
    Handler_discover 0
    Handler_read_first 4341235
    Handler_read_key 1042040870
    Handler_read_next 3640144465
    Handler_read_prev 91455243
    Handler_read_rnd 80305667
    Handler_read_rnd_next 3436239062
    Handler_rollback 0
    Handler_update 12138237
    Handler_write 31721612
    Key_blocks_not_flushed 0
    Key_blocks_unused 0
    Key_blocks_used 28995
    Key_read_requests 4270578688
    Key_reads 31892187
    Key_write_requests 21091301
    Key_writes 14781955
    Max_used_connections 651
    Not_flushed_delayed_rows 0
    Open_files 1910
    Open_streams 0
    Open_tables 1800
    Opened_tables 29276
    Qcache_free_blocks 15681
    Qcache_free_memory 61118976
    Qcache_hits 43766411
    Qcache_inserts 36015113
    Qcache_lowmem_prunes 77502
    Qcache_not_cached 256101
    Qcache_queries_in_cache 24464
    Qcache_total_blocks 65185
    Questions 111834925
    Rpl_status NULL
    Select_full_join 2209
    Select_full_range_join 0
    Select_range 11449190
    Select_range_check 0
    Select_scan 5609029
    Slave_open_temp_tables 0
    Slave_running OFF
    Slow_launch_threads 9675
    Slow_queries 135362
    Sort_merge_passes 9637
    Sort_range 10132761
    Sort_rows 291206874
    Sort_scan 217400
    Table_locks_immediate 95702220
    Table_locks_waited 779580
    Threads_cached 125
    Threads_connected 3
    Threads_created 26090
    Threads_running 1
    Uptime 3705896
    +--------------------------+------------+


    8- Only Vbulletin, there is another test Vbulletin in the server but hardly used.

    9- 400 – 800 users, Timeout is 900

    10 - http://www.omania.net/phpinfo.php (http://www.omania.net/phpinfo.php)

    11- Timeout 300
    KeepAlive On
    MaxKeepAliveRequests 500
    KeepAliveTimeout 5
    MinSpareServers 5
    MaxSpareServers 20
    StartServers 8
    MaxClients 450
    MaxRequestsPerChild 1000

    12- VB 3.1

    13- Nothing


  • I have just installed Eaccelerator 0.9.3 ..
    was easy thanks to Erwin ..
    I understand it will help when peak time comes :) ..


  • single P4 2.4GHZ cpu server is under powered to handle your peak times it seems as you're handling 651+ mysql concurrent connections to the mysql server which hit your preset max_connection limit of 650 already suggesting you may generate even more connections due to forum activity
    that kind of load is in the realm of really 2x dual cpu servers to handle the load!

    You are right, but I can not afford the costs :(
    I did the advised changes .. Thanks ..


    is there any tool to measure the server load all the day, as i can not be online at the peak time, and would like to see some kind of a log of the server load during the day , or the week.

    Cheers :cool:


  • well there's cacti but it's quite a bit of work to setup http://www.cacti.net/


  • single P4 2.4GHZ cpu server is under powered to handle your peak times it seems as you're handling 651+ mysql concurrent connections to the mysql server which hit your preset max_connection limit of 650 already suggesting you may generate even more connections due to forum activity

    that kind of load is in the realm of really 2x dual cpu servers to handle the load!

    But try the following first and see how it goes

    1. Upgrade to MySQL 4.1.14/15 check if ensim supports it

    2. Edit /etc/my.cnf and place the following mysql server settings in /etc/my.cnf and restart mysql server afterwards


    [mysqld]
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    skip-name-resolve
    safe-show-database
    skip-innodb
    max_connections = 650
    key_buffer = 32M
    myisam_sort_buffer_size = 64M
    join_buffer_size = 1M
    read_buffer_size = 1M
    sort_buffer_size = 2M
    table_cache = 1800
    thread_cache_size = 384
    wait_timeout = 180
    connect_timeout = 10
    tmp_table_size = 128M
    max_allowed_packet = 64M
    max_connect_errors = 10
    thread_concurrency = 2
    query_cache_limit = 1M
    query_cache_size = 64M
    query_cache_type = 1
    query_prealloc_size = 16384
    query_alloc_block_size = 16384

    [mysql.server]
    user=mysql
    basedir=/var/lib

    [mysqld_safe]
    err-log=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid
    open_files_limit = 8192

    [mysqldump]
    quick
    max_allowed_packet = 16M

    [myisamchk]
    key_buffer = 256M
    sort_buffer = 256M
    read_buffer = 128M
    write_buffer = 128M



    3. change httpd.conf from

    Timeout 300
    KeepAlive On
    MaxKeepAliveRequests 500
    KeepAliveTimeout 5
    MinSpareServers 5
    MaxSpareServers 20
    StartServers 8
    MaxClients 450
    MaxRequestsPerChild 1000

    to

    Timeout 300
    KeepAlive Off
    MaxKeepAliveRequests 120
    KeepAliveTimeout 5
    MinSpareServers 5
    MaxSpareServers 15
    StartServers 8
    MaxClients 200
    MaxRequestsPerChild 500

    restart apache

    try with keepalive off and on and see which is better







    • #If you have any other info about this subject , Please add it free.#
    Your name:
    E-mail:
    Telphone:

    Your comments:


    If you have any other info about any optimization suggestions :) , Please add it free.