htmlUnit介绍:

htmlunit 是一款开源的java 页面分析工具，读取页面后，可以有效的使用htmlunit分析页面上的内容。项目可以模拟浏览器运行，被誉为java浏览器的开源实现。是一个没有界面的浏览器，运行速度迅速。官方网址:

http://htmlunit.sourceforge.net/

htmlUnit示例:

pom依赖:

 <dependency>
            <groupId>net.sourceforge.htmlunit</groupId>
            <artifactId>htmlunit</artifactId>
            <version>2.26</version>
</dependency>

测试代码：

import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlInput;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import org.junit.Test;
 
import java.io.IOException;
 
 
public class BaiduDemo {
    @Test
    public void test() throws IOException {
        WebClient webClient = new WebClient();
        webClient.getOptions().setJavaScriptEnabled(false);
        webClient.getOptions().setCssEnabled(false);
        webClient.getOptions().setUseInsecureSSL(false);
 
 
        //获取页面
        String url ="https://www.baidu.com";
        HtmlPage page = webClient.getPage(url);
 
        System.out.println("页面文本:"+page.getTitleText());
 
 
        //获取页面元素
        HtmlInput htmlInput = page.getHtmlElementById("kw");
        System.out.println(htmlInput.asText());
        htmlInput.setValueAttribute("test");
 
        HtmlInput btn = page.getHtmlElementById("su");
        HtmlPage page2 = btn.click();
 
        System.out.println("页面2:"+page2.getTitleText());
    }
    
}

获取cookie:

BasicCookieStore cookieStore = new BasicCookieStore();
        Set<Cookie> cookies2 =webClient.getCookieManager().getCookies();
        for (Cookie cookie : cookies2) {
            cookieStore.addCookie(cookie.toHttpClient());
        }

通过htmlUnit获取的cookie信息发送https请求:

入参: 

cookieStore //通过htmlUnit获取的登陆信息

 url参数:要访问的页面参数

protected String sendRequest(BasicCookieStore cookieStoreString url) throws Exception {
        if (null == cookieStore) {
            return null;
        }
 
        //设置cookie
        HttpClientContext context = HttpClientContext.create();
        context.setCookieStore(cookieStore);
 
        //忽略SSL证书验证
 
        SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null new TrustStrategy() {
            //信任所有
            public boolean isTrusted(X509Certificate[] chain
                                     String authType) throws CertificateException {
                return true;
            }
        }).build();  
        CloseableHttpClient httpClient = HttpClients.custom().setSSLContext(sslContext).build();
        HttpGet httpGet = new HttpGet(url);
 
        //发送请求并获取response
        CloseableHttpResponse response = httpClient.execute(httpGet context);
        HttpEntity entity = response.getEntity();
 
        return EntityUtils.toString(entity);
    }

参考地址：http://blog.csdn.net/lifj07/article/details/8638098

来源：原创

来源：技术君