Citrix FAS – Notes from the Field

Reading Time: 12 minutes

Citrix Federated Authentication Service (FAS) is one of the most highly underrated features of the Citrix Virtual Apps and Desktop suite. FAS offers you modern authentication methods to your Citrix environment doesn’t matter if it is operated on-premises or running in the cloud. Citrix FAS is available since the XenApp/XenDesktop 7.9 release back in June 2016. If you still have not deployed FAS I would suggest you to take a closer look because it’s really a great solution which offers you totally new possibilities.

Why should you consider implementing Citrix FAS? Here are some of the use-cases:

– Seamless Token Enrollent for Azure MFA
– Microsoft ADFS Integration
– Simplify Account Management for Contractors
– Integration of AzureAD & Intune Devices
– Usage of FIDO and YubiKeys
– Integration of Identity Providers (IDPs) like Okta and Google

Whats important to know when you are planning to move to Citrix FAS: You need to let go of traditional authentication methods like the good old LDAP(S) or hopefully a combination of LDAP(S) & RADIUS. If we are speaking about modern authentication everything will be based on SAML or OAuth. Don’t get scared by these two technologies. After you have configured them for the first time it is easy as baking a cake and believe me: afterwards it is more complicated to create a LDAP server action on the Citrix ADC.

If you want to get started with configuring a Citrix FAS lab environment,  I can recommend you use Google as your Identity Provider. Why Google? Well some people (me included) don’t have an Azure subscription or want to stay on a low-budget for their home lab. With Google as your IDP you can create a fully working Citrix FAS environment and you don’t need to spend a penny. Sounds great isnt it?

This is not going to be a step-by-step manual on how to configure the FAS server or the Citrix ADC. If you are looking for this kind of information please visit the detailed post of Carl Stalhood. Carl will always know better than any of us 🙂

What I want to achieve with this blog post is to share my experiences from previous Citrix FAS implementations and give you some best practice approaches on your way. The first time is always the hardest and it can be kind of nerve wracking until everything is up and running.

Certificate Authority

The first important thing you need to know is that Citrix FAS is working with smart card authentication. This means we need to have a working Certificate Authority which is issuing the virtual smart cards. These smart cards are used during the login proccess of a user session. Please be aware that it needs to be a Microsoft Certificate Authority which is Active Directory integrated. Other Public Key Infrastructure solutions like OpenCA or OpenSSL are not working with the Citrix Federated Authentication Service. When installing the Citrix FAS service we are going to deploy three certificate templates.

Citrix_RegistrationAuthority_ManualAuthorization
This template will be used for creating the initial certificate signing request for the Citrix FAS server. The request is valid for 24 hours and needs to be manual approved by a CA administrator.

Citrix_RegistrationAuthority
As soon the previous request got approved the Citrix FAS server certificate is getting enrolled with this template. It will be used for generating CSRs for the virtual smart cards. The certificate is valid for 2 years and needs to manually renewed. If you miss the renewal the FAS service will stop working.

Citrix_SmartcardLogon
This template will issue the actual smart card which is going be used when logging into the VDA. The default lifetime is 7 days.

One of the questions which is coming up in every workshop with clients: Is it possible to change the name of the templates? Yes it is possible to change the name to a custom naming schema but you are going to lose the ability to manage the Citrix Federation Service with the Graphical User Interface. If you have the need for customized names the configuration can only be done via PowerShell.

Usually a public key infrastructure is divided into various Sub-CAs which are responsible for issuing different kind of certificates. Example:

Sub-CA #1: User Certificates
Sub-CA #2: Server Certificates
Sub-CA #3: Computer Certificates

My recommendation is to create a separate issuing CA for the Citrix FAS smart cards. This is not a must-have but from a security perspective this should be your preferred approach. Of course, this will depend on the knowledge of the CA administrator(s) and maybe takes additional time for arranging the prerequisites before starting with the actual FAS implementation.

For the deploying (1) and publishing (2) of the certificate templates and the following authorization of the Citrix Federation service you will need Domain- or Enterprise admin rights.

If the Citrix administrator(s) is not having the required domain rights for deploying the templates you need to pass the following PowerShell scripts to the the CA administrator. You will find them under the follwing path on the Citrix FAS server: “C:\Program Files\Citrix\Federated Authentication Service\CertificateTemplates”.

Import_Citrix_RegistrationAuthority.ps1

Import_Citrix_RegistrationAuthority_ManualAuthorization.ps1

Import_Citrix_SmartcardLogon.ps1

Custom Template Names

In case you want to change the name of the templates make a copy of the original template and apply your customizations. Example:

Now it is time to publish the templates.

During the creation of the Citrix FAS user rule we need to specify the StoreFront, VDAs and FAS users with a SID string. To make our life easier we are going to create the basic rule with the GUI and search for the objects in Active Directory. Afterwards we apply the necessary template/definition modifications with PowerShell.

Use the “Edit” button to set the correct permission regarding your Citrix infrastructure.

Create the custom certificate definition and assign it to the “CORP” FAS rule.

Security Hardening of the Certificate Templates

From a security perspective the built-in templates are shipped suboptimally and will create room for disscusions with the PKI team. I recommend you to apply the following changes.

Citrix_SmartcardLogon

1.) Change the Cryptogrpahy Key Size from 1024 to 2048 Bit

2.) Modify the Extended Key Usage (EKU) from “All” to “Smart Card Logon” only

Private Key Protection

The Citrix FAS server will store all the issued certificates in the registry. You will not find them in the Microsoft Certificate Store. It is possible to use a Hardware Security Module (HSM) or Trusted Platform Module (TPM) to store the private keys. One of your duties should be to keep the private keys as secure as possible. One of the approaches is to disable Remote Desktop Connections and restrict the access only to chosen FAS administrators.

Group Policy

Without a configured Group Policy (ADMX Templates needed) it is not possible to use the Citrix FAS console. Before starting the deployment make sure to create a GPO which will be linked to the Citrix FAS, Citrix Storefront and the VDAs. This can be a single policy which is linked to the different organization units. I will give a short explanation about the available options to configure.

Federated Authentication Service
List of all Citrix FAS Server using the FQDN
Example: FAS-01.lab.local

StoreFront FAS Rule
If we are working with custom template names we need to specify the name of the rule we created in the FAS console

In-Session Certificates
If you have authenticated to your VDI desktop and you are going to access published applications from a silo (session in session) this needs to enabled.

Citrix Delivery Controller

The only thing we need to configure on the Citrix Delivery Controller is to set the TrustRequestsSentToTheXmlServicePort to “true”. If you are already using a Citrix Gateway with Single-Sign-On to Citrix StoreFront this should already be enabled.

Citrix StoreFront

Create a dedicated store on Citrix StoreFront for FAS. After enabling the FAS functionality the store will always request a virtual smart card for the user. Even for logins which are only based on username and password. It is also recommended to synchronize the subscriptions between different stores to guarentee the best user experience –> Share Favorites/Subscriptions with Multiple Stores

Enable FAS for the created Store

It is important to work with a Callback URL when creating the Citrix Gateway on StoreFront. Citrix Federated Authentication Service will NOT work if you leave the Callback URL empty. You will be presented with the famous “Cannot complete your Request” message. When I was configuring Citrix FAS for the first time I was losing so much time because I was not aware of this.

Another potential stumbling block is to create a separate Callback Gateway (different VIP) and not use the same Gateway vServer wich the users are accessing from the outside of the company. We are going to have a SAML/OAuth policy on the Gateway which will redirect to our Identity Provider. This will break the Citrix StoreFront Callback process. Make sure to use a dedicated Callback Gateway! If you mess this up you will be presented with the “Cannot complete your request”.

If you are encountering this issue your Eventlog will show the follwing error:

Kerberos over HTTP

The Citrix StoreFront servers and the Virtual Desktop Agents are going to contact the Citrix FAS server using port 80 performing kerberos authentication. It is not possible to change the port/protocol to 443/SSL. If you are having security concerns: Kerberos is already encrypted and does not need any further hardening.

Domain Controller Certificate

Make sure all domain controllers are equipped with a “Domain Controller Authentication” certificate.  I have seen environments where the certificate was enrolled but when logging into the VDA you got left with the error “Request Not Supported”. If you are running into the issue just re-enroll the certificate and everything should start working.  See CTX218941 for further instructions.

DCOM

When the Citrix FAS server is communicating with the Microsoft CA this will happen via the DCOM protocol. The default port range is a random port from 1023 and above. In most environments the FAS/CA server will be located in a dedicated security zone and a random port range is not something the firewall administrators are going to let us get away with. For this particular case we can configure the DCOM service to a static port.

High Availability & Scalability

It is possible to deploy multiple Citrix FAS servers to achieve High Availablity and scalability in case one of the servers is having problems. So lets say we have deployed a single server setup. This means we are having one FAS server and CA which is providing the certificates. What’s happening if one of the systems crashes?

Scenario #1 – CA is not available anymore
– All the users who have logged into the FAS Store the previous 7 days will have a cached certificate on the Citrix FAS server and will be able to start their published resources
– If a user did not login to the FAS Store the last week, will not be able to connect to their apps and desktops

Scenario #2 – Citrix FAS is not available anymore
– No users will be able to start their published resources

Okay I think you understand, that we will need at least two Citrix FAS servers for redundancy so that we are not putting our environment in danger. A second certificate authority would be helpful to intercept the 7 days limitation if the user maybe was on holiday and we do not have a cached certificate on the FAS server.

If you are wondering how the load balancing is working when using more than one FAS server:
– The user certificates will not be generated/cached on all servers
– Citrix FAS is applying a hash algorithm to the User Prinicple Name
– The UPN will receive a primary/secondary/tertiary flag
– If the primary server has an outtage it will failover to the second server
– If the secondary server has an outtage it will failover to the teritary server

Some numbers about the scalability:
– 4 vCPUs up to 10.000 users (no cache)
– 8 vCPUs for 25.000 users (no cache)
– 1 FAS Server can handle 50.000 users if the certificates are pre-generated/cached

Pre-generate user certificates

Another thing you can do is to pre-generate user certificates on a regular basis. This will reduce logon times and the CA does not need to be high available because we already have all issued smart cards on the FAS server.

Needs to be executed on the Citrix FAS server.

Shadow Accounts

If we are going to federate users from different companies we need to create shadow accounts in our Active Directory. Lets assume  I am going to authenticate with my E-Mail “citrixguyblog@googlemail.com” to the Citrix Gateway. After the login was succesfull a smart card will be generated on the FAS server and I get redirected to Citrix StoreFront including SSO. Everything is good until this point, but my user is not seeing any published resources. That is the reason why you need to work with shadow accounts. We need to create the related UPN suffixes in the Active Directory Domain and Trusts console.

After we have created the “googlemail.com” sufix we can prepare the shadow account and publish the apps and desktops the users is allowed to use. You need to specify a password for the user account because it’s still a regular user object in Active Directory.

Google IDP

Before we can start configuring the Citrix ADC for OAuth to Google we need to create the OAuth client IDs. You can create them with a regular Google account.  https://console.developers.google.com/

Step 1 – Add your domain as an “Authorized Domain” under “OAuth consent screen”

Step 2 – Create the Client ID

Authorized JavaScript origins: https://apps.flashmob-saulgau.de
Authorized Redirect URLs: https://apps.flashmob-saulgau.de/oauth/login

Step 3 – Copy the Client ID and Client Secret

Citrix Gateway

Here are the CLI commands for configuring the Citrix Gateway for OAuth.

Troubleshooting

Troubleshooting can be quite challenging because many components are working together in a FAS environment. Here are some tips which will make your life easier.

1.) Use a Browser Addon to debug the SAML tokens (Firefox: SAML-tracer)

2.) Enable CAPI Logging
Path: HKLM:\CurrentControlSet\Services\crypt32
DiagLevel – DWORD – Value: 5
DiagMatchAnyMask – DWORD – Value: 0xffffff

3.) Enable Kerberos Logging
Path: HKLM:\CurrentControlSet\Control\Lsa\Kerberos\Parameters
LogLevel – DWORD – Value: 1
KerbDebuglevel – DWORD Value: 0xffffff

4.) Check the Domain Controller & VDA Security Eventlog

5.) Check Citrix FAS Eventlog

6.) Enable “Debug” Log  on Citrix ADC for analysing OAuth/SAML problems

Checklist for Deployment

1. Firewall Rules are implemented by the network team
2. Certificate Authority Design is defined
3. SSL Certificate for Citrix Gateway is available
4. Citrix FAS ADMX Templates have been copied to the PolicyDefinition share
5. FAS GPO configured and linked to StoreFront, FAS and VDAs
6. Published Certificate Templates
7. Security Hardening of Certificate Templates
8. Basic Installation of Citrix FAS
9. Auhorization of the Citrix FAS server
10. Created FAS User Rule
11. Create FAS Store on Citrix StoreFront
12. Enable FAS on the Store  (FASClaimsFactory, FASLogonDataProvider)
13. Create Citrix Gateway and allow Remote Access to FAS Store
14. Configure “Delegated Authentication” for Pass-Through from Citrix Gateway
15. Create Citrix Gateway  vServer
16. Create AAA Server vServer
17. Create OAuth/ADFS Profiles
18. Bind Policies/STAs to vServer

Summary

Hopefully I could gave you some insights into the Citrix Federated Authentication Service world and have encourage you to get started with your own implementation. If you have any questions or think there is room for improvments from your personal experience please let me know.

Thanks to my community friends Leee Jeffries and Daniel Weppeler for proofreading the article. Further credits go to Ben Splittgerber who helped me starting my FAS journey.