MariaDB 10.2 – Field ‘xxxxxxx’ doesn’t have a default value Error

I update one of my production server MariaDB 10.1 database server to 10.2 thinking everything will be ok, I did a live test in one of my VM machines and everything went smoothly…. well… on the outside that is.
As I finished updating the server, I started getting java.sql.SQLException: Field ‘xxxxxx’ doesn’t have a default value in my logs.
This didn’t happen before.
I soon found out from MariaDB document page….

Since MariaDB 10.2.4, SQL_MODE is by default set to NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO. In earlier versions of MariaDB 10.2, and since MariaDB 10.1.7, SQL_MODE is by default set to NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER. For earlier versions of MariaDB 10.1, and MariaDB 10.0 and before, no default is set.

So, this means that as soon as I updated my server to MariaDB 10.2, my sql_mode was set to “strict” mode by default internally.

The work around(?)/fix…go to the MariaDB config file (my.cnf) and change the sql_mode line to

sql_mode                = NO_ENGINE_SUBSTITUTION

Hope this helps those who ran into this problem.
(Note: Another fix is to manually set all table fields to have a default value – too long, too tiresome, Also you can change your code to set all default values – also too long and tiresome )

Link to other sources:
Here

MariaDB Unknown/unsupported storage engine: InnoDB

I had this problem re-occurring in one of my servers. MariaDB wont start due to this error.

To fix my problem, remove these log files…

/var/lib/mysqld/ib_logfile0
/var/lib/mysql/ib_logfile1
/var/lib/mysql/aria_log_control

ib_logfile may be 0 or 1

If you want to be safe.. just rename it to another file temporarily to see if it works.

Sources for the fix can be found Here and Here

Enable Ubuntu 16.04 SSH Tunneling at Boot Time

I wanted to connect to a remote server and setup a tunnel for my MariaDB replication ( yes I migrated to MariaDB ),
the only problem was that sometimes the net is erratic and can’t connect right away on boot time.
Found a code online to check for ping reply, if it does, continue with the tunneling.
I modified this to suit my needs.

In summary, the script checks for a ping reply from Google infinitely. Once it receives a response, it then creates the tunnel.
Source for the code here

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#!/bin/bash
((count = 0))
while [[ $count -ne 1 ]] ; do
    ping -c 1 8.8.8.8
    rc=$?
    if [[ $rc -eq 0 ]] ; then
        ((count = 1))                      # If okay, flag to exit loop.
    fi
 
done
 
if [[ $rc -eq 0 ]] ; then                  # Make final determination.
 
    screen -dm -S tunnel autossh -M 0 -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -p <PORT> <IP_ADDRESS> -L <LOCAL_TUNNEL_PORT>:localhost:<MYSQL_PORT>
 
 
    echo `date` >> log.txt
 
else
    echo "Tunnel Timeout...." `date` >> log.txt
fi

Regenerate Missing MariaDB/MySQL Config(my.cnf) File

I ran into this problem after upgrading/intalling MariaDB, the config file was missing.

 

To regenerate the config file… try executing…

1
dpkg --force-confmiss -i /var/cache/apt/archives/mysql-common*.deb

MySQL 5.6 and Up: High Memory Usage

Had a problem with a fresh install of MySQL 5.7 using more than normal memory.

Researching around the net, one feature that was added (5.6?) that fixed my problem was performance schema ( performance_schema ).

Disabling is easy enough as adding performance_schema=0 in my.cnf config file.

 

Thanks to Joel Abraham for the fix.

Connecting to MySQL on a Remote Server through Port Forwarding and Reverse SSH Tunnel

I had a problem connecting to a remote server that has its ports blocked by the ISP. So technically, I can’t port forward my way into the server. The only way around this is to SSH connect from my remote server to my main server and access the remote server via reverse SSH tunneling. This tutorial will show the steps on how I connected to my database server on the remote server via port forwarding on my PC and reverse SSH tunneling on my main server.

  1. Connect from the remote server (B) to the main server (A).
    • #> screen autossh -R22222:localhost:22 your_remote_IP_address
      • screen – new terminal so you can disconnect and work on other stuffs
      • autossh – reconnects if ssh is broken or is disconnected
      • -R -> tells the client that the remote port to open for the reverse tunnel
      • 22222 -> of course, the port to be used on the other end
      • localhost:22 -> anything attached on port 22222 will reach localhost port 22
      • your_remote_IP_address – not much explaining needed
  2. Now on the main server ( A ), to connect to remote server via the SSH tunnel created.
    • #> ssh -p 22222 username@localhost -L 4406:localhost:3306
      • -p 22222 -> instructs the SSH client to connect to localhost via port 22222
      • -L 4406:localhost:3306 -> this tells the client to forward all packets that is sent on port 4406 to localhost and target port 3306. That’s if the connection succeeds, will be on the remote server already.
  3. On my dev machine, I setup my SSH client ( putty ) to bind to port 99999 and forward  it to the main server on port 4406.
  4. I then have my DB Client attach to port 99999 on connect

Image shown below….

Sources: StackExchange

diagram

 

MySQL Slave Failed to Open the Relay Log

This problem is a little tricky, there are possible fixes that MySQL website has stated. Sad to say, the one’s I read in the forum and site didn’t fix my problem. What I encountered was that the relay-bin from my MySQL slave server has already been ‘rotated’, meaning deleted from the folder. This happens when the slave has been disconnected from the master for quite a long time already and has not replicated anything. A simple way to fix this is to flush the logs, but make sure the slave is stopped before using this command…

FLUSH LOGS;

Bring in a fresh copy of the database from the master-server and update the slave-server database. THIS IS IMPORTANT! Since if you don’t update the slave database, you will not have the data from the time you were disconnected until you reset the relay logs. So UPDATE YOUR SLAVE WITH THE LATEST DATABASE FROM THE MASTER!

Now when the logs are flushed,all the relay-bin logs will be deleted when the slave is started again. Usually, this fixes the problem, but when you start the slave and the failed relay log error is still there, now you have to do some more desperate measures… reset the slave. This is what I had to do to fully restore my MySQL slave server. Reseting the slaves restores all the settings to default… password, username, relay-log, port, table to replicate, etc… So better to have a copy of your settings first before actually do a slave reset. When your ready to rest the slave, do the command…

RESET SLAVE;

after which you should restore all your setting with a command something like…

CHANGE MASTER TO MASTER_HOST=.....

now start your server with…

SLAVE START;

check your slave server with…

SHOW SLAVE STATUS\G

look for …

Slave_IO_Running: Yes
Slave_SQL_Running: Yes

both should be YES, if not, check your syslog if there are other errors encountered. I’ll leave this until here since this is what I encountered and I was able to fix it.

Edit 5/14/11:

There is a possible change that after executing the CHANGE MASTER command that you’ll receive this error below…

ERROR 1201 (HY000): Could not initialize master info structure; more error messages can be found in the MySQL error log

This can occur when the relay logs under /var/lib/mysql were not properly cleaned and are still there. The next thing is to delete them manually, log back in to mysql, refresh logs, reset slave then execute the CHANGE MASTER command again. The file to delete would be relay-log.info .This should work now. Sometimes I don’t know why mysql can’t reset the slave logs.

MySQL: Query Real Values from Delimiter-Separated-String-IDs

I had a problem in using a string-comma-separated-value returned from a query in an “IN” statement. I used the IDs from one table, concat them into a comma separated value and insert them into another table. Baaaad idea… Now when I query that value, I can’t use it directly into an “IN” statement to retrieve their real values since its a string being returned from the query, the “IN” statement will not compare all the values inside as a set, but it will compare it as a string.

e.g.
SELECT value FROM my_table WHERE my_id IN (‘1, 2, 3’) is NOT equivalent to SELECT value FROM my_table WHERE my_id IN (‘1’, ‘2’, ‘3’)

So, if you have a table containing values 1 to 3, the first query will return only 1 while the second query will return all values; 1, 2 and 3.

I Googled around and found out that MySQL does not have a native equivalent of PHP’s explode() function. Crap… I had to do it the hard war and create a MySQL stored function to ‘explode’ the values from its delimiter, query the right value from the other table using the exploded IDs, concat them back together and return them as a string.

Below is the function I was able to patch together from different codes I found in the MySQL forum. I added comments below as to make things clearer. I named the function splitAndTranslate since that’s what I was really trying to implement. You can make up your own modifications and function name.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
DELIMITER //
DROP FUNCTION IF EXISTS `splitAndTranslate` //
CREATE FUNCTION splitAndTranslate(str TEXT, delim VARCHAR(124))
RETURNS TEXT
DETERMINISTIC
BEGIN
	DECLARE i INT DEFAULT 0;	-- total number of delimiters
	DECLARE ctr INT DEFAULT 0;	-- counter for the loop
	DECLARE str_len INT;		-- string length,self explanatory
	DECLARE out_str text DEFAULT '';	-- return string holder
	DECLARE temp_str text DEFAULT '';	-- temporary string holder
  	DECLARE temp_val VARCHAR(255) DEFAULT '';	-- temporary string holder for query
 
	-- get length
	SET str_len=LENGTH(str);	
 
	SET i = (LENGTH(str)-LENGTH(REPLACE(str, delim, '')))/LENGTH(delim) + 1;	
		-- get total number delimeters and add 1
		-- add 1 since total separated values are 1 more than the number of delimiters
 
	-- start of while loop
	WHILE(ctr<i) DO
		-- add 1 to the counter, which will also be used to get the value of the string
		SET ctr=ctr+1; 
 
		-- get value separated by delimiter using ctr as the index
		SET temp_str = REPLACE(SUBSTRING(SUBSTRING_INDEX(str, delim, ctr), LENGTH(SUBSTRING_INDEX(str, delim,ctr - 1)) + 1), delim, '');
 
		-- query real value and insert into temporary value holder, temp_str contains the exploded ID    		
		SELECT <real_value_column> INTO temp_val FROM <my_table> WHERE <table_id>=temp_str;
 
		-- concat real value into output string separated by delimiter
    		SET out_str=CONCAT(out_str, temp_val, ',');
	END WHILE;
	-- end of while loop
 
	-- trim delimiter from end of string
	SET out_str=TRIM(TRAILING delim FROM out_str);
	RETURN(out_str);	-- return 
 
END//

After creating the stored function, you can now use it normally like any MySQL function inside a query. So what I now do with the new function is…

SELECT splitAndTranslate( g.comma_separated_ids ) real_values FROM my_group_table g;

Thanks to Chris Stubben in the MySQL Forum, I used and modified his code to fit my need.