Skip to main content

Health Check of a Medulla server

This procedure allows you to quickly verify that a Medulla server and its main components are functioning properly.

In particular, it can be used:

  • after a server reboot;
  • after a maintenance operation;
  • after a Medulla update;
  • after a configuration change;
  • when a general platform malfunction is suspected.

Note:
The purpose of this Health Check is to quickly confirm that the main Medulla components are operational. If any of the checks fail, review the logs for the affected component before restarting or modifying the service.


1. Verify access to the Medulla interface

Start by logging in to the Medulla web interface.

In particular, verify that:

  • the login page is accessible;
  • authentication is working;
  • the main interface loads correctly;
  • the computers appear in the inventory.

Expected result:
The administrator can log in to Medulla and view the computer inventory as expected.


2. Check the status of the main Medulla services

Check the status of the main Medulla services using systemctl.

systemctl status ejabberd
systemctl status mmc-agent
systemctl status pulse-xmpp-agent-relay
systemctl status pulse2-register-pxe
systemctl status pulse2-package-server
systemctl status pulse-xmpp-master-substitute-master
systemctl status pulse-xmpp-master-substitute-logger
systemctl status pulse-xmpp-master-substitute-updates
systemctl status pulse-xmpp-master-substitute-assessor
systemctl status pulse-xmpp-master-substitute-inventory
systemctl status pulse-xmpp-master-substitute-deployment
systemctl status pulse-xmpp-master-substitute-monitoring
systemctl status pulse-xmpp-master-substitute-registration
systemctl status pulse-xmpp-master-substitute-subscription

For a quicker check, use:

systemctl is-active SERVICE_NAME

The expected result is:

active

Caution:
Depending on the Medulla installation and the enabled features, some services may not be present. The absence of an optional service does not necessarily indicate a malfunction.


3. Quickly identify services with errors

To find services that are currently failing:

systemctl --failed

If a Medulla service appears in this list, check its status:

systemctl status SERVICE_NAME

then its latest logs:

journalctl -u SERVICE_NAME -n 100

Tip:
Do not automatically restart a service that is failing before checking its logs. The messages present before the restart may be essential for identifying the cause of the incident.


4. Check ejabberd and XMPP

Medulla uses XMPP to communicate with its agents and several of its internal components.

Verify that the ejabberd service is running:

systemctl is-active ejabberd

The expected result is:

active

Next, check the number of users currently connected:

/usr/bin/ejabberdctl stats onlineusers

and the number of registered users:

/usr/bin/ejabberdctl stats registeredusers

To view the currently connected users:

ejabberdctl connected_users

Expected result:
ejabberd is active, and the expected Medulla agents and components are connected to XMPP.


5. Check MariaDB

Verify that the MariaDB service is running:

systemctl is-active mariadb

The expected result is:

active

Next, verify that MariaDB is responding correctly to queries:

mariadb -e "SELECT 1;"

The expected result includes:

1
1

6. Check the package server

Check the package server service:

systemctl is-active pulse2-package-server

The expected result is:

active

Also verify from the Medulla interface that an existing package is correctly visible.


7. Check the relay server

If the server also acts as a relay, check:

systemctl is-active pulse-xmpp-agent-relay

The expected result is:

active

In an architecture with multiple relays, also verify that the expected relays are accessible.

If a relay is unavailable, this can cause, among other things:

ABORT RELAY DOWN
ABORT ALTERNATIVE RELAYS DOWN

8. Check Syncthing

When Syncthing is used for package synchronization, check its status:

systemctl is-active syncthing@syncthing

The expected result is:

active

Caution:
An active Syncthing service alone does not guarantee that all packages are properly synchronized. If there is a transfer error or a missing package on a relay, also verify that synchronization has actually taken place.


9. Check disk space

Check the available space on the file systems:

df -h

Pay particular attention to the file system containing the deployment packages.

Important:
A full repository on a relay may prevent the synchronization or distribution of new packages without necessarily causing a service to stop.


10. Check server resources

Check available memory:

free -h

Next, check the server load:

uptime

For a real-time view:

top

Check the following in particular:

  • available memory;
  • swap usage;
  • system load;
  • whether there are any processes consuming an abnormal amount of resources.

11. Verify that a computer is communicating with Medulla

From the Medulla interface, select a computer known to be powered on and verify that it appears online.

This check provides a simple way to validate the communication chain:

Agent
  ↓
Réseau
  ↓
XMPP / ejabberd
  ↓
Medulla

If all computers appear offline even though they are powered on, first check for a system-wide communication or service issue.


12. Check the inventory

Select a test computer and verify that its inventory information is available in Medulla.

Check, for example:

  • the hostname;
  • the operating system;
  • network information;
  • installed software.

Expected result:
The computer's inventory information is available and consistent.


13. Test a deployment

After major maintenance or an update, it is recommended to perform a deployment on a test computer.

Preferably use a simple package that does not have a significant impact on functionality.

The deployment must result in:

DEPLOYMENT SUCCESS

This test validates, in particular:

  • communication with the agent;
  • relay assignment;
  • package availability;
  • the transfer;
  • execution on the target computer.

14. Check for recent errors

Even when all services are active, check the logs for recent errors.

For a service:

journalctl -u SERVICE_NAME -n 100

To view errors since the last startup:

journalctl -p err -b

Note:
A service may be active while still experiencing functional errors. Therefore, the systemd status is not always sufficient to confirm that the component is functioning properly.


15. Quick checklist

Check Expected result
Web interface Accessible
Authentication Functional
Medulla services Active
systemctl --failed No critical Medulla services are down
ejabberd Active
XMPP Components expected to be connected
MariaDB Active and accessible
Package server Active
Relay server Active and accessible
Syncthing Active if in use
Disk space Sufficient available space
Memory / load No overload
Test computer Online
Inventory Available
Test deployment DEPLOYMENT SUCCESS

16. What should I do if a check fails?

Problem detected Further troubleshooting
Medulla service is down Check systemctl status, then journalctl
ejabberd is unavailable Diagnose the XMPP service
Component missing from connected_users Diagnose its XMPP connection
MariaDB is unavailable View MariaDB status and logs
Relay unavailable Diagnose the relay server
Package missing from a relay Check Syncthing and synchronization
Computer offline See the FAQ “Why does a computer appear offline?”
Computer not registered See the FAQ “Why won’t a computer register?”
Deployment error Check the deployment status and audit

17. Summary

A Medulla server can be considered generally operational when:

  • the web interface is accessible;
  • the main services are active;
  • ejabberd is running;
  • the expected components are connected to XMPP;
  • MariaDB is running;
  • the relay servers are available;
  • the packages are accessible and synchronized;
  • system resources are not overloaded;
  • at least one computer is communicating normally with Medulla;
  • the inventory is working;
  • a test deployment can be performed successfully.

Recommendation:
After maintenance or a major update, don’t just check that the services are active. Also verify end-to-end functionality with an online computer, an inventory update, and a test deployment.