For a recent MISP installation I had to debug the reason why certain events were not pushed to a remote server. First a bit of context

• Both servers run the same version of MISP (a fairly recent version);
• Events are pushed from server A to server B. The synchronisation user used on server A existed on server B and had sufficient permissions;
• The server synchronisation was configured to push events if they were considered complete by the analyst. This is indicated by the workflow tag “state:complete”;
• Server A is on one network, whereas server B is on another external network;
• For testing purposes, we set the distribution level of our test even to ‘All communities’ (basically everybody).

Although MISP has a debug option (site_admin_debug) which can even give you SQL output, it does not contain debug information describing why a synchronisation action failed.

To debug my problem I had to use the advanced debugger options such as echo and var_dump, along with viewing the logs. Notice that I wrap these debug statement with comments so I can easily find them back afterwards (<= highly advised to do this if you've added a lot of debug statements).

Open one console where you tail the MISP error and work logs. I usually do this with

tail -f /var/www/MISP/app/tmp/logs/*

Next is adding our debug statements. There are various places where synchronisation can fail and it helps to understand the flow of how MISP publishes and pushes an event. For this session, we need to have a look in the Event Model, which you can find in app/Model/Event.php.

Pushing events after publishing is done in the function uploadEventToServersRouter

    public function uploadEventToServersRouter($id, $passAlong = null, $scope = 'events')

This function contains a foreach loop that checks every configured server. There are some additional verification steps but eventually it will upload the event with

$thisUploaded = $this->uploadEventToServer($event, $server, $HttpSocket, $scope);

This means we now have to look into the function uploadEventToServer.

public function uploadEventToServer($event, $server, $HttpSocket = null, $scope = 'events')

There you’ll find a call to execute the REST call (__executeRestfulEventToServer). Before you jump to this function, now is a good time to insert our first debug statement. If the code makes it to here, you already know that a failed synchronisation is not directly linked to the event metadata.

// DEBUG START
echo "Just before __executeRestfulEventToServer";
// DEBUG END
$result = $this->__executeRestfulEventToServer($event, $server, null, $newLocation, $newTextBody, $HttpSocket, $scope);
// DEBUG START
var_dump($result);
// DEBUG END

When I then edited the test event and published it, it returned this error message in the logs.

This made no sense at all as the event distribution level was set to ‘All communities’. Time to dive deeper and explore what’s in __executeRestfulEventToServer.

The first line of this function points us to yet another function: restfulEventToServer

$result = $this->restfulEventToServer($event, $server, $resourceId, $newLocation, $newTextBody, $HttpSocket, $scope);

The last two lines of this function are of importance. They create an HTTP socket and then the result from that HTTP socket is further parsed. This is our next opportunity to insert a debug statement.

// DEBUG START
echo "Before creating HTTP socket";
// DEBUG END
$response = $HttpSocket->post($uri, $data, $request);
// DEBUG START
var_dump($response);
// DEBUG END
return $this->__handleRestfulEventToServerResponse($response, $newLocation, $newTextBody);

The debug message in the logs was a bit of a surprise though …

The POST request part of the MISP synchronisation was being blocked by an application gateway! Because at the time of this debug-session the administrators of the application gateway were not available I can only assume that one of the verification rules is faulty.

This is the complete journey of this debug session: