Setup Huawei E1552 SMS Gateway in Raspbian using SMSTools3

Here, I was able to setup Huawei E1552 SMS Gateway in Raspbian using SMSTools3.

Requirements:

Steps:

  1. Install/Burn Raspbian to your microsd card. Instructions here.
  2. After burning/installing the image to your SD card. Install it to your Raspberry PI and configure it so you can SSH into it properly. Sadly, SSH daemon is not automatically configured to start on boot.
  3. To Enable SSH to autostart on boot… do
  4. :> update-rc.d ssh defaults
    :> update-rc.d ssh enable
  5. Restart and try to login remotely
  6. If you can not login, sshd may be configured to deny any remote root connection
  7. Got to /etc/ssh/ and edit sshd_config
  8. Look for this line…
  9. PermitRootLogin.......
  10. And change it to…
  11. PermitRootLogin yes

    Continue Reading

Ubuntu 16.04.02 Hang on Reboot or Shutdown

Had a problem after an update that when I force a restart or a shutdown on my server, it hangs on the last part of the process.

To fix my problem, I had to edit the /etc/default/grub config file and added acpi=force apm=power_off to the config line, between the double quotation marks…

1
GRUB_CMDLINE_LINUX_DEFAULT=""

and

1
GRUB_CMDLINE_LINUX=""

do a update-grub after saving then reboot one more time.. ( force sa restart when it hangs )
After the restart, try another reboot, it should work now without hanging.

Source of the fix is from Here

Heap Dump and Remote Tomcat 8-OpenJDK 8 Memory Monitoring

Memory problem.. the thorn of web apps… I have to connect to my remote server to monitor its memory usage and produce a heap dump.
Bellow is the steps I had to configure the server to allow me to connect a remote monitor tool.

Requirement: VisualVM app, download here

  1. Access your catalina.sh file.
  2. Look for JAVA_OPTS, if none found, you can add your own.
  3. Add …
    1
    
    -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/heap/dump/folder/

    to the line if found, if not, you can do

    1
    
    JAVA_OPTS="JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/to/heap/dump/folder/"

    The above code will force a heamp dump when the server encounter a out-of-mem exception.

  4. Next, to enable remote memory monitoring, add these commands after the line above, within the quotation mark, add proper spacing after each command…
    1
    2
    3
    4
    
    -Dcom.sun.management.jmxremote.port=<port_num>
    -Dcom.sun.management.jmxremote.ssl=false
    -Dcom.sun.management.jmxremote.authenticate=false
    -Djava.rmi.server.hostname=<host_ip_add>
  5. Now, run VisualVM and create a new JMX remote connection, under the CONNECTION field, add host_ip_address:port_num
    ex. 111.111.1.1:5555

Congrats! You should now be able to connect to your server and monitor your memory usage!

Enable Ubuntu 16.04 SSH Tunneling at Boot Time

I wanted to connect to a remote server and setup a tunnel for my MariaDB replication ( yes I migrated to MariaDB ),
the only problem was that sometimes the net is erratic and can’t connect right away on boot time.
Found a code online to check for ping reply, if it does, continue with the tunneling.
I modified this to suit my needs.

In summary, the script checks for a ping reply from Google infinitely. Once it receives a response, it then creates the tunnel.
Source for the code here

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/bin/bash
((count = 0))
while [[ $count -ne 1 ]] ; do
    ping -c 1 8.8.8.8
    rc=$?
    if [[ $rc -eq 0 ]] ; then
        ((count = 1))                      # If okay, flag to exit loop.
    fi
 
done
 
if [[ $rc -eq 0 ]] ; then                  # Make final determination.
 
    screen -dm -S tunnel autossh -M 0 -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -p <PORT> <IP_ADDRESS> -L <LOCAL_TUNNEL_PORT>:localhost:<MYSQL_PORT>
 
 
    echo `date` >> log.txt
 
else
    echo "Tunnel Timeout...." `date` >> log.txt
fi

Connecting to MySQL on a Remote Server through Port Forwarding and Reverse SSH Tunnel

I had a problem connecting to a remote server that has its ports blocked by the ISP. So technically, I can’t port forward my way into the server. The only way around this is to SSH connect from my remote server to my main server and access the remote server via reverse SSH tunneling. This tutorial will show the steps on how I connected to my database server on the remote server via port forwarding on my PC and reverse SSH tunneling on my main server.

  1. Connect from the remote server (B) to the main server (A).
    • #> screen autossh -R22222:localhost:22 your_remote_IP_address
      • screen – new terminal so you can disconnect and work on other stuffs
      • autossh – reconnects if ssh is broken or is disconnected
      • -R -> tells the client that the remote port to open for the reverse tunnel
      • 22222 -> of course, the port to be used on the other end
      • localhost:22 -> anything attached on port 22222 will reach localhost port 22
      • your_remote_IP_address – not much explaining needed
  2. Now on the main server ( A ), to connect to remote server via the SSH tunnel created.
    • #> ssh -p 22222 username@localhost -L 4406:localhost:3306
      • -p 22222 -> instructs the SSH client to connect to localhost via port 22222
      • -L 4406:localhost:3306 -> this tells the client to forward all packets that is sent on port 4406 to localhost and target port 3306. That’s if the connection succeeds, will be on the remote server already.
  3. On my dev machine, I setup my SSH client ( putty ) to bind to port 99999 and forward  it to the main server on port 4406.
  4. I then have my DB Client attach to port 99999 on connect

Image shown below….

Sources: StackExchange

diagram

 

Installing Gitorious in Ubuntu 12.04 Server – How-To

I wanted to setup my own Git repository locally and with an accompanying front end GUI. Before Gitorious, I installed Git from scratch and also installed Gitolite. That will be another post later on.

Initially I followed the script from http://blog.kyodium.net/2011/09/install-gitorious-on-ubuntu-1104.html but its based on Ubuntu 11.04, there are some missing packages that was not part of the installation script so I had to run the script manually and jump from one tutorial to the other, but the main flow I used is from that script. This is the documentation of the installation process.

Note: I added the PPA to Git’s updated source so I would be using their latest release not Ubuntu’s Git package. ( Google around for Git’s PPA repository, its easy enough 😉 )

Also I was not logged into root ( which I prefer, so I had to do a lot of sudo-ing =S )

Execute these line-by-line… please don’y copy everything then go pressing enter.

Update your Ubuntu server to latest patches and install the packages bellow.

sudo aptitude install build-essential zlib1g-dev libcurl4-openssl-dev postfix apg geoip-bin libgeoip1 sqlite3 libsqlite3-dev libgeoip-dev libpcre3 libpcre3-dev zlib1g zlib1g-dev libyaml-dev libmysqlclient-dev libonig-dev zip unzip memcached git-core git-svn git-doc git-cvs libreadline-dev openjdk-6-jdk sqlite3 libsqlite3-dev libmagick++3 libmagick++-dev libapache2-mod-xsendfile libxslt1-dev libreadline5 libxslt-dev libxml2-dev imagemagick apache2-dev activemq

Update Ruby Gems
Continue Reading

MySQL: Query Real Values from Delimiter-Separated-String-IDs

I had a problem in using a string-comma-separated-value returned from a query in an “IN” statement. I used the IDs from one table, concat them into a comma separated value and insert them into another table. Baaaad idea… Now when I query that value, I can’t use it directly into an “IN” statement to retrieve their real values since its a string being returned from the query, the “IN” statement will not compare all the values inside as a set, but it will compare it as a string.

e.g.
SELECT value FROM my_table WHERE my_id IN (‘1, 2, 3’) is NOT equivalent to SELECT value FROM my_table WHERE my_id IN (‘1’, ‘2’, ‘3’)

So, if you have a table containing values 1 to 3, the first query will return only 1 while the second query will return all values; 1, 2 and 3.

I Googled around and found out that MySQL does not have a native equivalent of PHP’s explode() function. Crap… I had to do it the hard war and create a MySQL stored function to ‘explode’ the values from its delimiter, query the right value from the other table using the exploded IDs, concat them back together and return them as a string.

Below is the function I was able to patch together from different codes I found in the MySQL forum. I added comments below as to make things clearer. I named the function splitAndTranslate since that’s what I was really trying to implement. You can make up your own modifications and function name.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
DELIMITER //
DROP FUNCTION IF EXISTS `splitAndTranslate` //
CREATE FUNCTION splitAndTranslate(str TEXT, delim VARCHAR(124))
RETURNS TEXT
DETERMINISTIC
BEGIN
	DECLARE i INT DEFAULT 0;	-- total number of delimiters
	DECLARE ctr INT DEFAULT 0;	-- counter for the loop
	DECLARE str_len INT;		-- string length,self explanatory
	DECLARE out_str text DEFAULT '';	-- return string holder
	DECLARE temp_str text DEFAULT '';	-- temporary string holder
  	DECLARE temp_val VARCHAR(255) DEFAULT '';	-- temporary string holder for query
 
	-- get length
	SET str_len=LENGTH(str);	
 
	SET i = (LENGTH(str)-LENGTH(REPLACE(str, delim, '')))/LENGTH(delim) + 1;	
		-- get total number delimeters and add 1
		-- add 1 since total separated values are 1 more than the number of delimiters
 
	-- start of while loop
	WHILE(ctr<i) DO
		-- add 1 to the counter, which will also be used to get the value of the string
		SET ctr=ctr+1; 
 
		-- get value separated by delimiter using ctr as the index
		SET temp_str = REPLACE(SUBSTRING(SUBSTRING_INDEX(str, delim, ctr), LENGTH(SUBSTRING_INDEX(str, delim,ctr - 1)) + 1), delim, '');
 
		-- query real value and insert into temporary value holder, temp_str contains the exploded ID    		
		SELECT <real_value_column> INTO temp_val FROM <my_table> WHERE <table_id>=temp_str;
 
		-- concat real value into output string separated by delimiter
    		SET out_str=CONCAT(out_str, temp_val, ',');
	END WHILE;
	-- end of while loop
 
	-- trim delimiter from end of string
	SET out_str=TRIM(TRAILING delim FROM out_str);
	RETURN(out_str);	-- return 
 
END//

After creating the stored function, you can now use it normally like any MySQL function inside a query. So what I now do with the new function is…

SELECT splitAndTranslate( g.comma_separated_ids ) real_values FROM my_group_table g;

Thanks to Chris Stubben in the MySQL Forum, I used and modified his code to fit my need.

Java 5 JDK and Jakarta Tomcat 5.5 in Ubuntu Server Installation

These are the steps I took to install Jakarta Tomkat 5.5 in conjunction with Java 2 SDK in Ubuntu server.

  1. Login as root, you can also use “su” though I prefer to login as root.
  2. apt-get install sun-java5-jdk
    • This will install Java 2 SDK (1.5), ubuntu will start downloading packages that are not found in your system.
  3. apt-get install tomcat5.5 tomcat5.5-admin tomcat5.5-webapps
    • Of course jakarta-tomcat 5.5, I’m using 5.5 since this supports java 1.5
  4. Now here’s the tricky part, location for Java and jakarta-tomcat
    • Java’s location is in /usr/lib/jvm/java-1.5.0-sun
    • Jakarta-Tomcat’s location is in /usr/share/tomcat5.5
      • We will need these location for the export configurations to be written in the .bashrc
  5. Now, open .bashrc in your favorite text editor, I used vim
  6. At the end of the file add these:

    export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun
    export CLASSPATH="/usr/share/tomcat5.5/common/lib/jsp-api.jar;/usr/share/tomcat5.5/common/lib/servlet-api.jar"
    export CATALINA_HOME=/usr/share/tomcat5.5
  7. Now save the file and exit, then log back in. This is required for the export script to be set.
  8. Before starting the server, the default port the installer set is 8180 instead of the 8080 port that Tomcat uses. To change this, go to /usr/share/tomcat5.5/conf/ and open the file server.xml. Locate the line:

    ...Connector port="8180" maxHttpHeaderSize="8192"...

    and change 8180 to 8080. Save and exit. Now your ready to test the server.
  9. Now try the server if it starts, /usr/share/tomcat5.5/bin/startup.sh You should see something like this:

    Using CATALINA_BASE: /usr/share/tomcat5.5
    Using CATALINA_HOME: /usr/share/tomcat5.5
    Using CATALINA_TMPDIR: /usr/share/tomcat5.5/temp
    Using JRE_HOME: /usr/lib/jvm/java-1.5.0-sun
  10. Shutting down the server is just /usr/share/tomcat5.5/bin/shutdown.sh
  11. Now, during installation of tomcat, Ubuntu automatically adds a startup and shutdown script for ubuntu to use as it starts up or shutsdown in /etc/init.d/ folder. Filename is tomcat5.5. As the server finishes starting up, you will notice tomcat isn’t running at all. One possible problem is that the script didn’t use the correct path for the java 2 sdk. You can check the boot log in /var/log/. Check if during boot up, tomcat5.5 encountered problems. For the problem stated above, you can fix this by going to /etc/default/ and edit the tomcat5.5 file. Locate the line:

    #JAVA_HOME=/usr/lib/j2sdk1.4-sun

    remove the ‘#’ and change it to…

    JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun

    or wherever your java directory is.After that, restart the server and see if tomcat was successfully started.

    /etc/init.d/tomcat5.5 status

Edit: Problem encountered:

  • Tomcat does not load in Ubuntu 8.04
    Found a little problem loading Tomcat 5.5 under Ubuntu 8.04, it either fails or white screen just appears. I don’t know what happened from 6.04 to 8.04 but once I compared the init.d/tomcat5.5 startup script from 6.04 and 8.04, there WERE changes to the scripts. I replaced the new tomcat5.5 script in 8.04 with the old one from 6.04 and the server started properly after a server restart. I don’t know why but this may have something to do with permissions and I have yet to trace the script.

  • java.lang.ClassNotFoundException: org.apache.jasper.tagplugins.jstl.If
    • The problem here is that the above name is incomplete… what it should have been is org.apache.jasper.tagplugins.jstl.core.*
    • Now, to fix the problem, locate /var/lib/tomcat5.5/webapps/jsp-examples/WEB-INF/tagPlugins.xml and open it with your text editor… I used vim. Now edit each line of code that has org.apache.jasper.tagplugins.jstl in it and add “.core” after jstl. What you will see after editing the code will be…
    • org.apache.taglibs.standard.tag.rt.core.IfTag

      org.apache.jasper.tagplugins.jstl.core.If

      org.apache.taglibs.standard.tag.common.core.ChooseTag

      org.apache.jasper.tagplugins.jstl.core.Choose

      org.apache.taglibs.standard.tag.rt.core.WhenTag

      org.apache.jasper.tagplugins.jstl.core.When

      org.apache.taglibs.standard.tag.common.core.OtherwiseTag

      org.apache.jasper.tagplugins.jstl.core.Otherwise

      org.apache.taglibs.standard.tag.rt.core.ForEachTag

      org.apache.jasper.tagplugins.jstl.core.ForEach

  • SEVERE: The scratchDir you specified: [folder_path] is unusable.
    • Sidenote: took me 2 days to figure out why this was happening (X_x)
    • Based from the Catalina logs, the folders under webapps was not accessible by tomcat5.
    • Now looking under all processes running ( ps command ), most bootup services were running under root (mysql, apache, etc ) but tomcat5 was not.
    • If I start tomcat5 when I logged into the console as root, the problem disappears. This lead me to believe this might have been the Tomcat5 user was not given the correct privileges.
    • Work-around 1: Elevate Tomcat5 user to root access level.
    • Work-around 2: Go to /etc/init.d/ and open Tomcat5.5 under any text editor ( I used vim ). Locate the line TOMCAT5_USER=tomcat55 and change it to TOMCAT5_USER=root . Restart your server or you can also execute the restart command for Tomcat5.5 in init.d folder

sources:

Friendster Script

Well, seems like a problem crept into a friend’s Friendster account. Everything in her page disappeared, or so it seems. The problem lay inside the shoutbox. A rouge script has inserted a small CSS script inside and made every “Div” element invisible. How? I dont know. I made this script for her to run under greasemonkey. This will rewrite what’s inside the shoutbox and let her rewrite whats inside back to normal.

The greasemonkey script can be found under the script page or here.