Create a new webapp or extend your existing Claims Mode, Windows Authenticated webapp. The new webapp should only have the “Enable Forms Based Authentication” checked. Here you must specify the membership and role provider names. You can choose any value here. If you choose values that are different from “smp” and/or “srp”, you will need to make adjustments in the web.configs of webapp, STS, and Central Administration.
Select “Custom Sign In Page and use the value “/_layouts/ASFSLoginPage.aspx” as shown below.
Once created, you will need to create a root site collection. This is important if you want Client Integration to work properly. See kb2590564.
Now you are ready to install and configure Shibboleth SP.