缘起
首先，用检查Referer的方式来防披御CSRF并不是很好的方法。因项目临时有需要，所以做为过渡方案。
为什么判断referer不是很好的办法？
- referer 可能为空
https跳转http没有referer
https跳转不同的域名的https没有referer
通过特殊构造的POST请求没有referer
一些的proxy会把referer去掉
用户直接在浏览器里访问（GET请求）
- 判断的逻辑复杂（用正则匹配？）
- 友站中招，殃及池鱼
- 可以作为过渡方案，非长久之计
构造空referer请求的一些参考资料
防御CSRF目前比较好的办法是CSRF Token，参考另一篇blog：Cookie & Session & CSRF。
##收集资料
先搜索下前人有没有这类相关的工作。
搜索到的关于RefererFilter的信息并不多。
不过这里学到了一些东东：
https://svn.apache.org/repos/asf/sling/tags/org.apache.sling.security-1.0.0/src/main/java/org/apache/sling/security/impl/ReferrerFilter.java
- 是否允许localhost, 127.0.0.1这样referer的请求？
- 是否允许本地的IP/host的请求？
再搜索下java里提取request的referer的方法，还有filter里重定向请求的方法。
再仔细看了下OWASP的文档：
https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet
##确定方案
- 默认拦截“POST|PUT|DELETE|CONNECT|PATCH”的请求
- HttpServletRequest里提取到referer
- 用java.net.URL来提取referer里的host
- 判断host是否符合要求，支持完全匹配的域名和子域名
- 不符合要求的请求回应403或者重定向到指定的页面
为什么不用正则的方式来处理referer？
- 正则表达式通常比较慢
- 很难判断一个复杂的正则表达式是否真的正确
- URL是很复杂的，不要手动处理URL，参考URL的语法
##思考需要提供的配置项
实际最终提供了这些配置项，考虑到像host这样的配置不是经常变动的，所以没有提供从外部配置文件加载配置的功能。
matchMethods 即拦截的方法，默认值"POST|PUT|DELETE|CONNECT|PATCH"，通常不用配置
allowSubDomainHosts 匹配子域名，以"|"分隔，如"test.com|abc.com"，
则http://test.com, http://xxx.test.com这样的请求都会匹配到，推荐优先使用这个配置
completeMatchHosts 完全匹配的域名，以"|"分隔，如"test.com|abc.com"，则只有http://test.com 这样的请求会匹配
像http://www.test.com 这样的请求不会被匹配
responseError 被拦截的请求的response的返回值，默认是403
redirectPath 被拦截的请求重定向到的url，如果配置了这个值，则会忽略responseError的配置。
比如可以配置重定向到自己定义的错误页： /referer_error.html
bAllowEmptyReferer 是否允许空referer，默认是false，除非很清楚，否则不要改动这个
bAllowLocalhost 是否允许localhost, 127.0.0.1 这样的referer的请求，默认是true，便于调试
bAllowAllIPAndHost 是否允许本机的所有IP和host的referer请求，默认是false
##编码的细节
- 重定向时，注意加上contextPath
response.sendRedirect(request.getContextPath() + redirectPath);
- 构造URL时，非法的URL会抛出RuntimeException，需要处理
##正确地处理URL
感觉这个有必要再次说明下：
http://docs.oracle.com/javase/tutorial/networking/urls/urlInfo.html
用contain, indexOf, endWitch这些函数时都要小心。
public static void main(String[] args) throws Exception {
URL aURL = new URL("http://example.com:80/docs/books/tutorial"
+ "/index.html?name=networking#DOWNLOADING");
System.out.println("protocol = " + aURL.getProtocol());
System.out.println("authority = " + aURL.getAuthority());
System.out.println("host = " + aURL.getHost());
System.out.println("port = " + aURL.getPort());
System.out.println("path = " + aURL.getPath());
System.out.println("query = " + aURL.getQuery());
System.out.println("filename = " + aURL.getFile());
System.out.println("ref = " + aURL.getRef());
}
##用curl来测试
最后用curl来做了一些测试：
curl --header "Referer:http://test.com" http://localhost:8080/filter-test/referer curl -X POST --header "Referer:http://test.com" http://localhost:8080/filter-test/referer curl -X POST --header "Referer:xxxxx" http://localhost:8080/filter-test/referer curl -X POST http://localhost:8080/filter-test/referer curl -X POST --header "Referer:http://abc.test.com" http://localhost:8080/filter-test/referer curl -X POST --header "Referer:http://abc.hello.com.test.com" http://localhost:8080/filter-test/referer
##实现的代码
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* <pre>
* 支持的配置项：
* matchMethods 即拦截的方法，默认值"POST|PUT|DELETE|CONNECT|PATCH"，通常不用配置
* allowSubDomainHosts 匹配子域名，以"|"分隔，如"test.com|abc.com"，
* 则http://test.com, http://xxx.test.com这样的请求都会匹配到，推荐优先使用这个配置
* completeMatchHosts 完全匹配的域名，以"|"分隔，如"test.com|abc.com"，则只有http://test.com 这样的请求会匹配
* 像http://www.test.com 这样的请求不会被匹配
*
* responseError 被拦截的请求的response的返回值，默认是403
* redirectPath 被拦截的请求重定向到的url，如果配置了这个值，则会忽略responseError的配置。
* 比如可以配置重定向到自己定义的错误页： /referer_error.html
* bAllowEmptyReferer 是否允许空referer，默认是false，除非很清楚，否则不要改动这个
* bAllowLocalhost 是否允许localhost, 127.0.0.1 这样的referer的请求，默认是true，便于调试
* bAllowAllIPAndHost 是否允许本机的所有IP和host的referer请求，默认是false
*
* {@code
* <filter>
* <filter-name>refererFilter</filter-name>
* <filter-class>com.test.RefererFilter</filter-class>
* <init-param>
* <param-name>completeMatchHosts</param-name>
* <param-value>test.com|abc.com</param-value>
* </init-param>
* <init-param>
* <param-name>allowSubDomainHosts</param-name>
* <param-value>hello.com|xxx.yyy.com</param-value>
* </init-param>
* </filter>
*
* <filter-mapping>
* <filter-name>refererFilter</filter-name>
* <url-pattern>/*</url-pattern>
* </filter-mapping>
* }
* </pre>
*
* @author hengyunabc
*
*/
public class RefererFilter implements Filter {
static final Logger logger = LoggerFactory.getLogger(RefererFilter.class);
public static final String DEFAULT_MATHMETHODS = "POST|PUT|DELETE|CONNECT|PATCH";
List<String> mathMethods = new ArrayList<>();
boolean bAllowEmptyReferer = false;
boolean bAllowLocalhost = true;
boolean bAllowAllIPAndHost = false;
/**
* when bAllowSubDomain is true, allowHosts is "test.com", then
* "www.test.com", "xxx.test.com" will be allow.
*/
boolean bAllowSubDomain = false;
String redirectPath = null;
int responseError = HttpServletResponse.SC_FORBIDDEN;
HashSet<String> completeMatchHosts = new HashSet<String>();
List<String> allowSubDomainHostList = new ArrayList<String>();
@Override
public void init(FilterConfig filterConfig) throws ServletException {
mathMethods.addAll(getSplitStringList(filterConfig, "matchMethods", "\\|", DEFAULT_MATHMETHODS));
completeMatchHosts.addAll(getSplitStringList(filterConfig, "completeMatchHosts", "\\|", ""));
List<String> allowSubDomainHosts = getSplitStringList(filterConfig, "allowSubDomainHosts", "\\|", "");
completeMatchHosts.addAll(allowSubDomainHosts);
for (String host : allowSubDomainHosts) {
// check the first char if is '.'
if (!host.isEmpty() && host.charAt(0) != '.') {
allowSubDomainHostList.add("." + host);
} else {
allowSubDomainHostList.add(host);
}
}
responseError = getInt(filterConfig, "responseError", responseError);
redirectPath = filterConfig.getInitParameter("redirectPath");
bAllowEmptyReferer = getBoolean(filterConfig, "bAllowEmptyReferer", bAllowEmptyReferer);
bAllowLocalhost = getBoolean(filterConfig, "bAllowLocalhost", bAllowLocalhost);
if (bAllowLocalhost) {
completeMatchHosts.add("localhost");
completeMatchHosts.add("127.0.0.1");
completeMatchHosts.add("[::1]");
}
bAllowAllIPAndHost = getBoolean(filterConfig, "bAllowAllIPAndHost", bAllowAllIPAndHost);
if (bAllowAllIPAndHost) {
completeMatchHosts.addAll(getAllIPAndHost());
}
}
@Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException,
ServletException {
if (servletRequest instanceof HttpServletRequest && servletResponse instanceof HttpServletResponse) {
HttpServletRequest request = (HttpServletRequest) servletRequest;
HttpServletResponse response = (HttpServletResponse) servletResponse;
String method = request.getMethod();
/**
* if method not in POST|PUT|DELETE|CONNECT|PATCH, don't check
* referrer.
*/
if (!mathMethods.contains(method.trim().toUpperCase())) {
filterChain.doFilter(request, response);
return;
}
String referrer = request.getHeader("referer");
boolean bAllow = false;
if (isBlank(referrer)) {
bAllow = bAllowEmptyReferer;
} else {
URL url = null;
try {
url = new URL(referrer);
String host = url.getHost();
if (completeMatchHosts.contains(host)) {
bAllow = true;
} else {
for (String domain : allowSubDomainHostList) {
if (host.endsWith(domain)) {
bAllow = true;
break;
}
}
}
} catch (RuntimeException e) {
logger.error("illegal referrer! referrer: " + referrer, e);
bAllow = false;
}
}
if (bAllow) {
filterChain.doFilter(request, response);
return;
} else {
if (isBlank(redirectPath)) {
response.sendError(HttpServletResponse.SC_FORBIDDEN);
} else {
response.sendRedirect(request.getContextPath() + redirectPath);
}
}
} else {
filterChain.doFilter(servletRequest, servletResponse);
}
}
@Override
public void destroy() {
}
private static boolean isBlank(CharSequence cs) {
int strLen;
if (cs == null || (strLen = cs.length()) == 0) {
return true;
}
for (int i = 0; i < strLen; i++) {
if (Character.isWhitespace(cs.charAt(i)) == false) {
return false;
}
}
return true;
}
private static boolean getBoolean(FilterConfig filterConfig, String parameter, boolean defaultParameterValue) {
String parameterString = filterConfig.getInitParameter(parameter);
if (parameterString == null) {
return defaultParameterValue;
}
return Boolean.parseBoolean(parameterString.trim());
}
private static int getInt(FilterConfig filterConfig, String parameter, int defaultParameterValue) {
String parameterString = filterConfig.getInitParameter(parameter);
if (parameterString == null) {
return defaultParameterValue;
}
return Integer.parseInt(parameterString.trim());
}
/**
* <pre>
* getSplitStringList(filterConfig, "hosts", "\\|", "test.com|abc.com");
*
* if hosts is "hello.com|google.com", will return {"hello.com", google.com"}.
* if hosts is null, will return {"test.com", "abc.com"}
* </pre>
*
* @param filterConfig
* @param parameter
* @param regex
* @param defaultParameterValue
* @return
*/
private static List<String> getSplitStringList(FilterConfig filterConfig, String parameter, String regex, String defaultParameterValue) {
String parameterString = filterConfig.getInitParameter(parameter);
if (parameterString == null) {
parameterString = defaultParameterValue;
}
String[] split = parameterString.split("\\|");
if (split != null) {
List<String> resultList = new LinkedList<String>();
for (String method : split) {
resultList.add(method.trim());
}
return resultList;
}
return Collections.emptyList();
}
public static Set<String> getAllIPAndHost() {
HashSet<String> resultSet = new HashSet<String>();
Enumeration<NetworkInterface> interfaces;
try {
interfaces = NetworkInterface.getNetworkInterfaces();
while (interfaces.hasMoreElements()) {
NetworkInterface nic = interfaces.nextElement();
Enumeration<InetAddress> addresses = nic.getInetAddresses();
while (addresses.hasMoreElements()) {
InetAddress address = addresses.nextElement();
if (address instanceof Inet4Address) {
resultSet.add(address.getHostAddress());
resultSet.add(address.getHostName());
} else if (address instanceof Inet6Address) {
// TODO how to process Inet6Address?
// resultSet.add("[" + address.getHostAddress() + "]");
// resultSet.add(address.getHostName());
}
}
}
} catch (SocketException e) {
logger.error("getAllIPAndHost error!", e);
}
return resultSet;
}
}
##其它的一些东东
在浏览器里如何访问IPV6的地址？
用”[]”把IPV6地址包围起来，比如localhost的：
http://[::1]
参考：
- http://superuser.com/questions/367780/how-to-connect-a-website-has-only-ipv6-address-without-domain-name
- https://msdn.microsoft.com/en-us/library/windows/desktop/ms740593(v=vs.85).aspx
原文地址：http://hengyunabc.github.io/how-to-create-a-referer-filter/
