Archive for May, 2013

Simple Example of Java JSON Encoding in Servlet

May 17th, 2013

Here is a simple example showing how to output json in a servlet. I’ll use the “JSON-java” package which is recommended by http://json.org/java/ and can be downloaded at https://github.com/douglascrockford/JSON-java. To encode Java objects into a JSON string, we simply use org.json.JSONObject and org.json.JSONArray.java.

import org.json.JSONArray;
import org.json.JSONObject;

/**
 * A simple example of showing how to output json in a Servlet.
 */
@WebServlet(description = "Test servlet showing how to output json in Java", urlPatterns = { "/services/testjson" })
public class TestServlet extends HttpServlet {
	private static final long serialVersionUID = 1L;
	
	protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
	    
		response.setContentType("application/json");
		response.setHeader("Cache-Control", "nocache");
        	response.setCharacterEncoding("utf-8");
		PrintWriter out = response.getWriter();

		JSONObject json = new JSONObject();

		// put some value pairs into the JSON object as into a Map.
		json.put("status", 200);
		json.put("msg", "OK");

		// put a "map" 
		JSONObject map = new JSONObject();
		map.put("key1", "val1");
		map.put("key2", "val2");
		json.put("map", map);
		
		// put an "array"
		JSONArray arr = new JSONArray();
		arr.put(5);
		arr.put(3);
		arr.put(1);
		json.put("arr", arr);

		// finally output the json string		
		out.print(json.toString());
	}
}

There is another package called json-simple which is also very handy and simple. More information about json-simple can be found at https://code.google.com/p/json-simple/.

Install Lucene Solar with Tomcat on Windows

May 7th, 2013

Installing the example solar web application is actually super easy. Here we suppose you already installed the Tomcat sevlet container on your computer TOMCAT_HOME (for example: D:\prog\apache-tomcat-7.0.35). If not, you can download it here http://tomcat.apache.org and install it (at TOMCAT_HOME).

The next step is to install Solar:

  1. download Solr at: http://lucene.apache.org/solr/
  2. extract the solar package into, for example, solr-4.3.0
  3. copy the sample “Solr Home” directory solr-4.3.0\example\solr\ (Note that it’s the folder ‘example\solr\’ itself, not the content under it!!!) into the home directory of tomcat: TOMCAT_HOME if you start tomcat with Tomcat Monitor or into TOMCAT_HOME\bin if you start tomcat with TOMCAT_HOME\bin\startup.bat. Note that the solr hoe directory must be put in the right directory. This is very important!!! Otherwise you will get such an exception: “HTTP Status 500 – {msg=SolrCore ‘collection1′ is not available due to init failure: Could not load config for solrconfig.xml …… ”
    In a word, the solr home directory should be put in the Java current working directory. You can also “Configure the servlet container such that a JNDI lookup of “java:comp/env/solr/home” by the Solr webapp will point to the Solr home”.
  4. copy the solr war file (solr-N.N.N.war) under solr-4.3.0\dist\ as solr.war into the tomcat webapps directory TOMCAT_HOME/webapps. Tomcat will automatically deploy it.
  5. After solr.war is extracted, copy the SLF4j logging jars from solr-4.3.0/example/lib/ext into TOMCAT_HOME/lib or TOMCAT_HOME/webapps/solr/WEB-INF/lib. If this step is ignored, this exception will be thrown out: org.apache.catalina.core.StandardContext filterStart
    SEVERE: Exception starting filter SolrRequestFilter
    org.apache.solr.common.SolrException: Could not find necessary SLF4j logging jars. If using Jetty, the SLF4j logging jars need to go in the jetty lib/ext directory. For other containers, the corresponding directory should be used. For more information, see: http://wiki.apache.org/solr/SolrLogging

OK. That’s it! Now locate your web browser to http://localhost:8080/solr (Change the port if necessary) and you will see the admin page of solr.

If there are exceptions, check your tomcat logs (such as localhost.YYYY-MM-DD.log) under TOMCAT_HOME\logs\.

PS. actually to install solar on windows is not so much different from that on any other OS. :)

More details can be found on the Solr Wiki: http://wiki.apache.org/solr/SolrInstall

Java with UTF-8 encoding (especially with Chinese)

May 5th, 2013

Here are just some memos on how to deal with non-unicode characters (like Chinese) in java.

  • Decode characters from URL in a servlet:
    String s = request.getParameter("mykeywords");
    s = new String(s.getBytes("ISO-8859-1"), "UTF-8");

    But if the uri encoding is specified in server.xml as:
    <Connector URIEncoding="UTF-8" connectionTimeout="20000" port="8080" protocol="HTTP/1.1" redirectPort="8443"/>,
    there is no need to decode it. Simply use:
    String s = request.getParameter("mykeywords"); and s will be utf-8 encoded.

  • Send response with UTF-8:
    response.setCharacterEncoding("UTF-8");
  • Connect mysql with UTF-8:
    private static final String DB_URL = "jdbc:mysql://DB_HOST:3306/DB_SCHEMA?useUnicode=true&characterEncoding=utf8";
  • To encode JSP files with UTF-8, put this at the beginning of a JSP file:
    <%@ page contentType="text/html;charset=UTF-8" %>
  • With SpringMVC, for GET requests, set <Connector URIEncoding="UTF-8" connectionTimeout="20000" port="8080" protocol="HTTP/1.1" redirectPort="8443"/>,.
    And for POST requests, set this as the first filter in web.xml:
    <filter>
    <filter-name>encodingFilter</filter-name>
    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
    <init-param>
    <param-name>encoding</param-name>
    <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
    <param-name>forceEncoding</param-name>
    <param-value>true</param-value>
    </init-param>
    </filter>
  • And finally, make sure every file itself is stored as UTF-8.

problem with multi fastcgi_finish_request requests in nginx

May 1st, 2013

The php server nginx has a very handy function fastcgi_finish_request(). If fastcgi_finish_request is called in a function, the function will return immidiately but still do the job in background. So it is intuitive to think of using fastcgi_finish_request() to do some jobs in parallel when needed.

However, when a process call the fastcgi_finish_request() many times in a row, the later calls have to wait for the previous call to finish. So it’s not such a big help when you call fastcgi_finish_request() many times in a row, because all these requests will be executed in sequence. Rather you should put all the long calls in a fastcgi_finish_request() call.

For example:

function a() {
     long_work(); // 1st call
     // if we write a log here, we'll find 1st call immediately returns.
     long_work(); // 2nd call
     // if we write a log here, we'll find this 2nd call will have to wait for 10s before it returns.
}

function long_work() {
    sleep(5);
    fastcgi_finish_request();
}

A workaround to mimic multithread with fastcgi_finish_request() is to use curl. We put the long work in a php file, and then use curl to request the php file.

// ------ long_work.php ----------
long_work();
function long_work() {
    sleep(5);
    fastcgi_finish_request();
}


And then in function a(), we can request long_work many times.

function a() {
    // curl( 'http://localhost/long_work.php' ); // 1st call
    // curl( 'http://localhost/long_work.php' ); // 2nd call
}