It seems that the standard uploader does not check the allowed filesize. Can I check the filesize and avoid that it is getting stored on the server?

Yes, please copy the following code into the Begin Submit Piece:

$this->execPieceByName('ff_InitLib');
// get the name of the uploaded file element
$filename = ff_getSubmit('upload');
// check size of uploaded file
if ($filename && filesize($filename) >= 1048576)
{
     // 1MB
     // set a custom error code
     $this->status = _FF_STATUS_UPLOAD_FAILED;
     // delete uploaded file
     unlink($filename);
     header("Location: URLtoForm&error=true");
}

Change URL to Form to the URL that links to the first page of your form. Don't change '&error=true'

Copy this code into the Before Form Piece of the Form:

$this->execPieceByName('ff_initLib');
$show_error = JRequest::getVar('error');
if ($show_error == 'true')
{
     echo '<script type="text/javascript">alert("Error_Message");</script>';
}

Change 'Error_Message' to your desired error message.

Now the form should redirect the user to the first page of the form and show a PopUp message.

Note that this does not work for forms in IFrames!

Beside this PHP solution there are many workarounds in the internet which only use Javascript.

Please check  http://stackoverflow.com/questions/3717793/javascript-file-upload-size-validation for more information.