Configuring Kerberos/AD Authentication For The SMB Server

From FileSys.Org Wiki

By default the JFileServer SMB server uses NTLM authentication, for more secure and single signon authentication you can configure the SMB server to use Kerberos logons.

This document details how to configure the JFileServer, and JFileServer Enterprise, SMB server to use Kerberos authentication using a Windows Active Directory (AD) server setup.

Setting Up The File Server User Account

The JFileServer SMB server requires an AD user account to map a service principal name to that the client will request when accessing the SMB server.

Create a user account on the AD server, fill in the user logon name eg. jfileserver, disable the 'User must change password at next logon' option, and enable the 'Password never expires' option. Set a password for the new account.

In the user account properties enable the 'Do not require Kerberos pre-authentication' option.

The file server host also requires a DNS entry with the AD server.

Generate The Keytab and Service Principal Name

Generate a key table for the file server using the ktpass command.

ktpass -princ cifs/<file-server-host-name>.<domain>@<REALM>
 -pass <file-server-account-password>
 -mapuser <domain>\<file-server-account-name>
 -crypto RC4-HMAC-NT
 -out <output-file-name>
 -kvno 0

The generated key table will be used by the SMB server to authenticate the server so it will then be able to authenticate the client logons.

Copy the key table to the file server host system.

Create the service principal name mapping, that clients will use to connect to the SMB file server service, using the setspn utility.

setspn cifs/<file-server-host-name> <file-server-account-name>
setspn cifs/<file-server-host-name>.<domain> <file-server-account-name>

Configure Kerberos On The File Server

The Java authentication APIs require a Kerberos configuration file, this can either be in the default location such as /etc/krb5.conf on linux and macOS, C:\winnt\krb5.ini on Windows, or the location can be specified on the Java command line using the property.

A sample Kerberos configuration file is shown below, the Kerberos realm is always specified in uppercase.

default realm = FILESYS.ORG

  kdc =
  admin_server =

[domain-realm] = FILESYS.ORG = FILESYS.ORG

Setup The Java Login Configuration

The Java login configuration is used by the file server application to logon the server application using Kerberos by using the details in the key table that was generated on the Windows AD server.

The Java login configuration file can be stored in your user home folder or a system folder such as /etc. Create a new file called jfileserver.login.conf. The default Java login configuration section name is FileServerSMB. A sample Java login configuration file is shown below :-

FileServerSMB { required

Note: Use spaces in the Java login configuration file, tab characters cause problems.

To enable the Java login configuration the main Java security configuration file needs to be updated. This is usually located in the <JRE-folder>/lib/security/ file.

Look for the example configuration line that starts with #java.login.config.url.1=.... Uncomment the line, or add a new line, similar to the value shown:-


Configure The JFileServer SMB Authentication

In the JFileServer XML configuration file enable the SMB server Kerberos authentication via the <authenticator> configuration section within the <fileserver> <SMB> section of the configuration. A minimal sample configuration section is shown below:-


    <authenticator type="enterprise">

The following configuration tags are available for the SMB authenticator:-

Configuration Value Description
<useSPNEGO/> Force Simple Protected Negotiation to be used for the authentication, this will be selected automatically if Kerberos authentication is enabled. If only NTLM authentication is enabled the default is to use NTLMSSP in the authentication exchanges.
<mode>...</mode> Server mode, should always be 'USER'
<Realm>...</Realm> Kerberos realm, always uppercase
<LoginEntry<...</LoginEntry> Specifies the Java login configuration entry name if the default name of 'FileServerSMB' is not used
<Password>...</Password> Kerberos file server account password. Not required
<disallowNTLMv1/> Do not allow the weaker NTLM v1 logons, only NTLM v2 logons will be accepted
<disableNTLM/> Only allow Kerberos authentication. NTLM authentication will not be advertised by the server or accepted
<Debug/> Enable authentication debug output
<kerberosDebug/> Enables the Java authentication API debug output. Equivalent of setting the Java system properties '' and ''