Oracle Linux Virtualization Manager (OLVM), Hard Partitioning and SSO error with olvm-vmcontrol

Oracle Linux Virtualization Manager is one of the few remaining options for performing hard partitioning to restrict the use of Oracle licenses on a system.

In this case, a customer already had an (outdated) OLVM environment with three servers, each with two CPU licenses for the Oracle database, for a total of six CPU licenses, which corresponds to the use of twelve cores on the Intel/AMD environment. Due to hardware replacement cycles, the existing OLVM was not upgraded but reinstalled.

The environment in the new setting now consists of two nodes, each with 8 cores, for a total of 16, and an OLVM engine on an additional VM (in VMWare). In order to avoid having to add additional licenses for the Oracle database and since the company already has the necessary expertise in-house, the requirement was to limit the CPU cores or threads when moving the virtual machines (VMs) to the new OLVM environment. 

This is usually relatively easy to do by following the few steps in the document https://www.oracle.com/a/ocom/docs/linux/ol-kvm-hard-partitioning.pdf.

olvm-vmcontrol must be installed on the engine host, and then CPU pinning to the threads can be performed using a command with olvm-vmcontrol.

Unfortunately, however, the customer encountered an error when executing olvm-vmcontrol: Either it said that the user admin@internal could not connect due to incorrect credentials, or that admin@ovirt, which is actually used on the engine interface, did not exist at all:

'Error during SSO authentication “access_denied”: “Cannot authenticate user No valid profile found in credentials..”'

However, the problem was not an incorrect password, but rather that KeyCloak was suggested as the default during installation (never change default values, if unsure is not always a good advice). KeyCloak is still “experimental” and is responsible for preventing olvm-vmcontrol from connecting to the engine.

You can easily check whether KeyCloak is installed on the host running the engine:
grep KEYCLOAK_BUNDLED /etc/ovirt-engine/engine.conf.d/12-setup-keycloak.conf

If the value is “True,” you will definitely encounter the problem that olvm-vmcontrol cannot log in to the OLVM engine. Fortunately, there is a solution – KeyCloak must be disabled. Since this involves several steps, I refer you to the relevant post in Oracle's support portal under note KB495706 (simply enter this in the search field at the new portal of support.oracle.com).

Once disabling has been done, olvm-vmcontrol can then login with the user admin@internal and the CPU/thread binding to the VM can be configured as desired. For subsequent installations, the customer now knows that KeyCloak configuration should not be set to "Yes" when configuring the engine at the beginning.

In order to achieve the most distributed utilization of the CPU NUMAs possible, when pinning the threads, care should be taken to ensure that they are distributed as evenly as possible across the available NUMAs (in this case 2). VMs not intended for the operation of Oracle licenses can then be distributed across the remaining threads.

However, pinning the CPUs has one disadvantage: live migrations of a VM from one node to another are no longer supported. This means that the VM must be stopped, the definition edited (hosts), and the VM restarted. This is a (small organizational/high availability) price to pay in return for the license savings. 

Install RLWRAP on Oracle Database Appliance without internet connection

 If you need to install RLWRAP on an Oracle Database Appliance (ODA) with actual releases (19.26-19.28 ) you need to install the following rpms. As we use rlwrap a lot also in our Robotron DBA tool set, we install it nearly everywhere. 

While it is easy to install rlwarp with a working internet connection and the ODAs yum/dnf repository configuration, most of our customers don't allow the ODAs access to the public internet (except of ASR). 

Here is the list of the rpms you need to install (ordered) from the public yum base repository:
- ncurses-c++-libs: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/ncurses-c++-libs-6.1-10.20180224.el8.x86_64.rpm
- ncurses-devel: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/ncurses-devel-6.1-10.20180224.el8.x86_64.rpm
- readline-devel: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/readline-devel-7.0-10.el8.x86_64.rpm
- readline: should be already installed, if not: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/readline-7.0-10.el8.x86_64.rpm
- m4: https://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/m4-1.4.18-7.el8.x86_64.rpm

The following must be installed using the rpms from the Appstream folder. 
- autoconf: https://yum.oracle.com/repo/OracleLinux/OL8/appstream/x86_64/getPackage/autoconf-2.69-29.el8_10.1.noarch.rpm
- automake: https://yum.oracle.com/repo/OracleLinux/OL8/appstream/x86_64/getPackage/automake-1.16.1-8.el8.noarch.rpm
- perl-file-slurp: https://yum.oracle.com/repo/OracleLinux/OL8/appstream/x86_64/getPackage/perl-File-Slurp-9999.19-19.el8.noarch.rpm

rlwrap can either be installed from oracle yum epel oder fedoraproject repository
- rlwrap from oracle yum epel : https://yum.oracle.com/repo/OracleLinux/OL8/developer/EPEL/x86_64/getPackage/rlwrap-0.46.2-3.el8.x86_64.rpm
- rlwrap from fedoraproject: https://dl.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/r/rlwrap-0.46.2-3.el8.x86_64.rpm  

If you install rlwrap from the fedoraproject und you get an error "GPG check FAILED" you need to download the public key RPM-GPG-KEY-EPEL-8  from https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8, save it, e.g. as RPM-GPG-KEY-EPEL-8.txt and import it with "rpm --import RPM-GPG-KEY-EPEL-8.txt" before installing the rpm.
  
If you patch an ODA you might get an error at the prepatchreport for the server. If so, uninstall automake, autoconf and m4 with dnf remove <package> and re-install them after you finished the ODA patch (server and storage). 
  
If you download everything locally and upload/put all rpms into the /tmp folder of the ODA, you can run dnf install /tmp/<name-of-the-rpm> and install them one after the other. 

My Oracle Support portal changed to worse. How to find at least some DocIDs you still have...

Well, we all looked forward to a new support portal starting at Dec. 7th, enabling us to use AI and to work better and faster than before. What really happend? Links are broken, all bookmarks gone (in my case roundabout 100 sorted for different technical products and problems, patches, white papers), the patch search does not work like it did before, and so on, and so on... At the end, it is not better than before - it is a lot more worse. 

At least for all the links in my blog I hope I have found a solution. Typically, all blogs do not tell you the document title, but you can see the DocID. 
As the DocID has changed into a "KB" with a number, they still seem to be used in the background. If you do know the DocID which consists of a number, followed by a "Dot" and a second number, you may get access by using the following procedure. 

First is, login to your support.oracle.com environment. Then, put the old DocID without the "Dot" and everything following into this URL, replacing <SHORT_DOC_ID> with the value you have. 

https://support.oracle.com/support/?anchorId=&kmContentId=<SHORT_DOC_ID>&page=sptemplate&sptemplate=km-article

One of my favorite notes is 888.1, the title is something with "Oracle Database", but if you search for "Oracle Database primary note", well, you don't find it in the search results. 

E.g. to get the DocID 888.1 you can use 

https://support.oracle.com/support/?anchorId=&kmContentId=888&page=sptemplate&sptemplate=km-article

The title by the way is now "(KB106822) Primary Note for Database Quarterly Release Updates".  If you look at the first lines, you can see the following text 

"This was MOS Document ID: 888.1 in Legacy MOS.  Post migration, we will be moving to KB888 (Date TBA)"

What the f*... 

Ok, back to another note I really need a lot - the last version of OPatch was at DocID 293369.1. Again, changing the link to 

https://support.oracle.com/support/?anchorId=&kmContentId=293369&page=sptemplate&sptemplate=km-article

leads to the Primary Note for OPatch. 

I'm still mourning the lost bookmarks, but at least for the most documents I find at the internet, I may have found a solution to bring them back without struggling with the new search functionality. Unfortunately it seems to not work with all DocIDs, some do show a "404" error, but most do work. 

Please comment if this works for the things you need or if you still struggle. 


Oracle database - if (not) exists clause is backported to 19c

 You may have read my blog post about the new feature of the if (not) exists clause available in database 23ai HERE, where a detailed description can be found. This is one of the most wanted features for a lot of (old fashioned) developers waiting for 23ai on-premises (which is still not available yet). 

The good news, this feature was now back-ported to 19c!
After the installation of the Oracle Database Release Update 19.28 you are able to use this feature in 19c. 

You don't need to change the compatible parameter of your database, it can still stays on 19.0.0.


You can see at the SQL logon message that my database is running on 19.28 and I am able to use the new if (not) exists syntax:


Like at 23ai, the table jso_test is first created with a column with a length of 10. The second - new - create table would create the table with a column with a length of 20. As "if not exists" is specified, it does not throw an error, BUT the length of the column is still 10. As before, one need to add an alter table to modify this column to a length of 20 characters. 
The third attempt is like it was until 19.27 where the if (not) exists clause did not exists. It throws an error. 

Now you can start changing your database scripts to use the if (not) exits clause, but don't forget to alter the columns to reflect schema changes. And you don't have to wait for 23ai on-premises!


Very good news for all database developers scripting the schema changes.