When the server uses VNC Authentication, after the Challege/Response, the server sends a status code to indicate whether authentication worked. See RFB Doc v. 3.8 section 6.1.3.
An integer indicating the status of authentication: 0 = OK; 1 = Failed; 2 = Too Many (deprecated).