本篇主要是搜集了日常开发中比较常用的几个工具类,他们均以Java实现,包含了比较常用的方法。比如Java实现http常见的get和post请求,Java实现文件内容读写,Java实现处理空间地理信息数据,比如合并四维空间数据的处理等。
将这些工具类记录下来,便于以后涉及类似的场景和业务,可以直接取用和参考,也利于日后不断的改进和优化。
- package com.sina.weibo.util;
-
- import java.io.*;
- import java.net.HttpURLConnection;
- import java.net.URL;
-
- /**
- * @classname HttpClientUtil
- * @author bingqing5
- * @date 2022/08/04 17:16
- * @version 2.0
- */
-
- public class HttpClientUtil {
-
- /**
- * 以get方式发起http请求,调用接口,获取数据
- *
- * @param httpUrl URL参数
- * @return 返回调用接口获取的字符串数据
- */
- public static String doGet(String httpUrl) {
- // 结果字符串
- String result = "";
- try {
- // 创建远程url连接对象
- URL url = new URL(httpUrl);
- StringBuilder document = new StringBuilder();
- // 将远程url转换为http远程连接类
- HttpURLConnection conn = (HttpURLConnection) url.openConnection();
- // 设置请求方式:get
- conn.setRequestMethod("GET");
- conn.setDoOutput(true);
- conn.setDoInput(true);
- // 设置URLConnection的连接参数
- conn.setRequestProperty("accept", "*/*");
- conn.setRequestProperty("connection", "keep-Alive");
- // 设置连接主机服务器的超时时间
- conn.setConnectTimeout(15000);
- // 设置读取远程返回的数据时间:60000毫秒
- conn.setReadTimeout(60000);
- InputStream input = null;
- try {
- // 建立远程连接
- conn.connect();
- input = conn.getInputStream();
- } catch (Exception e) {
- e.printStackTrace();
- }
-
- BufferedReader reader = new BufferedReader(new InputStreamReader(input,"utf-8"));
- String str = "";
- while ((str = reader.readLine()) != null) {
- document.append(str);
- }
- result = document.toString();
- reader.close();
- // 释放资源
- input.close();
- // 关闭连接
- conn.disconnect();
-
- } catch (Exception e) {
- e.printStackTrace();
- }
-
- return result;
- }
-
- /**
- * 以post方式发起请求,调用接口,获取数据
- * @param url 接口链接
- * @param param 请求参数
- * @return 以字符串形式返回接口获取的数据
- */
- public static String doPost(String url, String param) {
- PrintWriter out = null;
- BufferedReader in = null;
- String result = "";
- try {
- URL realUrl = new URL(url);
- // 将远程url转换为http远程连接类
- HttpURLConnection conn = (HttpURLConnection) realUrl.openConnection();
- // 设置请求属性
- conn.setRequestProperty("accept", "*/*");
- conn.setRequestProperty("connection", "keep-Alive");
- conn.setRequestProperty("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;SV1)");
- conn.setRequestProperty("Charset", "UTF-8");
- // 设置URL请求方式
- conn.setRequestMethod("POST");
- // 使用URL进行输出
- conn.setDoOutput(true);
- // 使用URL进行输入
- conn.setDoInput(true);
-
- // 建立输出流,并写入数据
- out = new PrintWriter(conn.getOutputStream());
- out.print(param);
- out.flush();
- in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
- String line;
- while ((line = in.readLine()) != null) {
- result += line;
- }
- } catch (Exception e) {
- e.printStackTrace();
- }
- finally {
- try {
- if (out != null) {
- out.close();
- }
- if (in != null) {
- in.close();
- }
- } catch (IOException e) {
- e.printStackTrace();
- }
- }
-
- return result;
- }
-
- }
以上HttpClientUtil类提供了两个方法:doGet()和doPost()基本满足用Java实现调用第三方提供的接口做数据处理的场景。本人在日常工作中使用这两个方法去调用PHP后台接口,获取的数据一般是JSON字符串。经历了不知道多少次的调用,这两个方法都工作的很好。
- package com.sina.weibo.util;
-
- import java.io.*;
- import java.util.ArrayList;
- import java.util.List;
-
- /**
- * {@code FileUtils}是文件处理工具类。它提供了多种方法用于处理文件
- * 的读取和写入。
- *
- * @className FileUtils
- * @author bingqing5
- * @date 2022/03/07 11:30
- * @version 2.0
- */
-
- public class FileUtils {
-
- /**
- * 文件内容追加,以换行符结尾
- *
- * @param fileName 文件名(绝对文件路径名)
- * @param content 文件内容
- */
- public static void appendInfoToFile(String fileName, String content) {
- File file = new File(fileName);
- try {
- // 如果文件不存在,新建一个文件
- if (!file.exists()) {
- file.createNewFile();
- }
- OutputStreamWriter fileWriter = new OutputStreamWriter(new FileOutputStream(file, true), "UTF-8");
- content = content + "\r\n";
- fileWriter.write(content);
- fileWriter.flush();
- fileWriter.close();
- } catch (IOException e) {
- e.printStackTrace();
- }
- }
-
- /**
- * 文件内容读取,将内容存入集合中返回
- *
- * @param fileName 文件名(绝对文件路径名)
- * @return 返回字符串list集合
- */
- public static List
readInfoFromFile(String fileName) { - File file = new File(fileName);
- if (!file.exists()) {
- return null;
- }
- List
list = new ArrayList(); - try {
- BufferedReader bufferedReader = new BufferedReader(new FileReader(file));
- String str = null;
- while (null != (str = bufferedReader.readLine())) {
- list.add(str);
- }
- bufferedReader.close();
- } catch (IOException e) {
- e.printStackTrace();
- }
- return list;
- }
-
-
- }
以上FileUtils工具类提供了两个方法:appendInfoToFile()和readInfoFromFile(),分别用于将数据写入文件和将数据文件中读取存放到list集合中。需要注意的时,实践中,如果数据量达上亿条,1个G以上,读取时间会稍微拉长,达一分钟以上。当然,也并不一定要将数据存入list集合中再进行处理,可以读取一行数据以后及时处理,再读取下一行。但是往往在数据量很大的时候,处理数据需要关注一个问题:那就是如何防止数据重复处理,特别是在程序运行过程中被中断以后。将数据读取一遍存入list集合中个,诚然会消耗额外的内存资源,但与此同时也可以使用hashSet或LinkedHashSet存放已处理数据,在读取过程中与数据源集合中的数据对比,以便于达到避免重复处理相同的数据带来的开销。
readInfoFromFile()方法只是针对特定文件目录下的单一文件,现实中也有很多场景需要读取文件目录下同类型的所有文件,这就需要去扫描文件目录下后缀相同的文件。以下方法,提供了实现:
- package siweiAutoUpdate.parse;
-
- import java.io.File;
- import java.io.FileInputStream;
- import java.io.FileOutputStream;
- import java.io.InputStream;
- import java.io.InputStreamReader;
- import java.io.Reader;
- import java.util.ArrayList;
- import java.util.HashMap;
- import java.util.HashSet;
- import java.util.List;
- import java.util.Map;
- import java.util.Set;
-
- public class ParseSiweiBaseData {
-
- private static List
filePNameList = new ArrayList(); - private static List
filePOIList = new ArrayList(); - private static List
fileRelationList = new ArrayList(); -
-
- private static void scanPNameFile(File file) {
- if (file.isDirectory()) {
- File[] files = file.listFiles(new FileFilter() {
- public boolean accept(File pathname) {
- return pathname.isDirectory()
- || (pathname.getPath().endsWith(".mid") && pathname.getName().startsWith("PName"));
- }
- });
- for (File one : files) {
- scanPNameFile(one);
- }
- } else {
- filePNameList.add(file.getAbsolutePath());
- }
- }
-
- private static void scanPOIFile(File file) {
- if (file.isDirectory()) {
- File[] files = file.listFiles(new FileFilter() {
- public boolean accept(File pathname) {
- return pathname.isDirectory() || (pathname.getPath().endsWith(".mid")
- && !pathname.getName().startsWith("POI_") && pathname.getName().startsWith("POI"));
- }
- });
- for (File one : files) {
- scanPOIFile(one);
- }
- } else {
- filePOIList.add(file.getAbsolutePath());
- }
- }
-
- private static void scanRelationFile(File file) {
- if (file.isDirectory()) {
- File[] files = file.listFiles(new FileFilter() {
- public boolean accept(File pathname) {
- return pathname.isDirectory()
- || (pathname.getPath().endsWith(".mid") && pathname.getName().startsWith("POI_Relation"));
- }
- });
- for (File one : files) {
- scanRelationFile(one);
- }
- } else {
- fileRelationList.add(file.getAbsolutePath());
- }
- }
-
- }
GetoUtils类相关依赖:
- <dependency>
- <groupId>com.vividsolutionsgroupId>
- <artifactId>jtsartifactId>
- <version>1.8version>
- dependency>
代码如下:
- package com.sina.weibo.util;
-
- import com.vividsolutions.jts.geom.*;
- import com.vividsolutions.jts.io.ParseException;
- import com.vividsolutions.jts.io.WKTReader;
-
- import java.io.*;
- import java.text.DecimalFormat;
- import java.util.zip.GZIPInputStream;
-
- public class GeoUtils {
-
- public static final ThreadLocal
FORMAT_LNGLAT = new ThreadLocal(){ - @Override
- protected DecimalFormat initialValue() {
- // TODO Auto-generated method stub
- return new DecimalFormat("###.00000");
- }
- };
-
- /**
- * 创建WGS84的GeometryFactory
- *
- * WGS-84 地心坐标系 4326
- * GCJ-02火星坐标系 3857
- *
- *
- * 地理坐标系:GCS_WGS_1984:WKID: 4326 权限: EPSG
- 投影坐标系:WGS_1984_Web_Mercator_Auxiliary_Sphere:WKID: 3857 权限: EPSG
- *
- *
- */
- static final GeometryFactory geometryFactory = new GeometryFactory(
- //new PrecisionModel(PrecisionModel.FLOATING), 4326);
- new PrecisionModel(PrecisionModel.FLOATING), 3857);
-
- /**
- * 获得WGS84的GeometryFactory
- * @return WGS84的GeometryFactory
- */
- public static final GeometryFactory getGeometryFactory() {
- return geometryFactory;
- }
-
- public static double formatLngLat(double lnglat){
- return Double.valueOf(FORMAT_LNGLAT.get().format(lnglat));
- }
-
- /**
- * 根据WKT生成MultiPolygon
- * @param wktPolygon 多边形的WKT
- * @return WKT生成MultiPolygon
- * @throws ParseException 生成MultiPolygon时发生异常
- */
- public static final MultiPolygon getMultiPolygon(String wktPolygon) {
- String wktPolygonN=null;
- try{
- if(wktPolygon.toUpperCase().startsWith("POLYGON")){
- wktPolygonN=wktPolygon.replace("POLYGON", "MULTIPOLYGON").replace("((", "(((").replace("))", ")))");
- }else{
- wktPolygonN=wktPolygon;
- }
- WKTReader reader = new WKTReader(geometryFactory);
- return (MultiPolygon) reader.read(wktPolygonN);
- }catch(Exception e){
- e.printStackTrace();
- }
- return null;
- }
-
- /**
- * 通过经纬度点生成多边形对象
- * @param ordinates 经纬度点,格式:经度,纬度,经度,纬度...
- * @return 生成的多边形对象
- */
- public static final Polygon getPolygon(double[] ordinates) {
- if (ordinates.length % 2 == 1)
- throw new RuntimeException("ordinates.length = " + ordinates.length + " error!");
-
- int lnglatLength = ordinates.length / 2;
-
- // 保证首位经纬度相同
- Coordinate[] coordinates = null;
- if (ordinates[0] == ordinates[ordinates.length - 2] && ordinates[1] == ordinates[ordinates.length -1]) {
- coordinates = new Coordinate[lnglatLength];
- } else {
- coordinates = new Coordinate[lnglatLength + 1];
- coordinates[coordinates.length - 1] = new Coordinate(ordinates[0], ordinates[1]);
- }
-
- for (int i = 0; i < lnglatLength; i++) {
- coordinates[i] = new Coordinate(ordinates[i*2],ordinates[i*2+1]);
- }
- LinearRing lr = geometryFactory.createLinearRing(coordinates);
- return geometryFactory.createPolygon(lr, null);
- }
-
- /**
- * 通过经纬度点生成路线
- * @param ordinates 经纬度点,格式:经度,纬度,经度,纬度...
- * @return 生成的路线
- */
- public static final LineString getLineString(double[] ordinates) {
- if (ordinates.length % 2 == 1)
- throw new RuntimeException("ordinates.length = " + ordinates.length + " error!");
-
- Coordinate[] coordinates = new Coordinate[ordinates.length / 2];
- for (int i = 0; i < coordinates.length; i++) {
- coordinates[i] = new Coordinate(ordinates[i*2],ordinates[i*2+1]);
- }
-
- return geometryFactory.createLineString(coordinates);
- }
-
- /**
- * 通过经纬度点生成多边形对象
- * @param coordinates 经纬度点
- * @return 生成的多边形对象
- */
- public static final Polygon getPolygon(Coordinate[] coordinates) {
-
- // 保证首位经纬度相同
- Coordinate[] lrCoordinates = null;
- if (coordinates[0].x == coordinates[coordinates.length -1].x && coordinates[0].y == coordinates[coordinates.length -1].y) {
- lrCoordinates = coordinates;
- } else {
- lrCoordinates = new Coordinate[coordinates.length + 1];
- for (int i = 0; i < coordinates.length; i++) {
- lrCoordinates[i] = coordinates[i];
- }
- lrCoordinates[lrCoordinates.length - 1] = coordinates[0];
- }
-
- LinearRing lr = geometryFactory.createLinearRing(lrCoordinates);
- return geometryFactory.createPolygon(lr, null);
- }
-
- /**
- * 根据WKT生成空间对象
- * @param
空间对象的类 - * @param wkt WKT
- * @param t 空间对象的实例
- * @return WKT生成空间对象
- * @throws ParseException 生成MultiPolygon时发生异常
- */
- public static final
T getGeometry(final String wkt, final Class t) throws ParseException { - WKTReader reader = new WKTReader(geometryFactory);
- return (T) reader.read(wkt);
- }
-
- /**
- * 根据WKT生成MultiLineString
- * @param wktLineString WKT
- * @return 生成的MultiLineString
- * @throws ParseException 生成MultiLineString时发生异常
- */
- public static final MultiLineString getMultiLineString(String wktLineString) throws ParseException {
- WKTReader reader = new WKTReader(geometryFactory);
- return (MultiLineString) reader.read(wktLineString);
- }
-
- /**
- * 地球赤道半径,单位米
- */
- public static final double EARTH_RADIUS = 6378137;
-
- /**
- * 面积(平方米)
- *
- */
- public static double getArea(Geometry wktGeo){
- double km=wktGeo.getArea()*10000;//平方公里
- return km*1000000;
- }
-
- /**
- * 距离(米)
- *
- */
- public static long getDistance(Geometry geometry1,Geometry geometry2){
- long distance=(long)(geometry1.distance(geometry2)*100000);
- return distance;
- }
-
-
- /**
- * 角度转弧度
- *
- * */
- public static double angleToRad(double angle) {
- return angle * Math.PI / 180.0;
- }
-
- /**
- * 弧度转角度
- *
- * */
- public static double radToAngle(double rad) {
- return 180.0/Math.PI * rad;
- }
-
- /**
- * 计算两个空间坐标点之间的距离,单位米
- * @param lng1 空间坐标点1的经度,单位度
- * @param lat1 空间坐标点1的纬度,单位度
- * @param lng2 空间坐标点2的经度,单位度
- * @param lat2 空间坐标点2的纬度,单位度
- * @return 两个空间坐标点之间的距离,单位米
- */
- public static double computeDistance(double lng1, double lat1, double lng2, double lat2) {
- final double radLat1 = angleToRad(lat1);
- final double radLat2 = angleToRad(lat2);
- final double a = radLat1 - radLat2;
- final double b = angleToRad(lng1) - angleToRad(lng2);
- double s = 2 * Math.asin(Math.sqrt(Math.pow(Math.sin(a / 2), 2) +
- Math.cos(radLat1) * Math.cos(radLat2) * Math.pow(Math.sin(b / 2), 2)));
- s = s * EARTH_RADIUS;
- //s = Math.round(s * 10000) / 10000;
- return s;
- }
-
- /**
- * 获得p2点相对于p1点的极坐标方向,方向范围[0,360)
- * @param lng1 p1点的经度,单位度
- * @param lat1 p1点的纬度,单位度
- * @param lng2 p2点的经度,单位度
- * @param lat2 p2点的纬度,单位度
- * @return 获得p2点相对于p1点的极坐标方向,方向范围[0,360)
- */
- public static double getDirection(double lng1, double lat1, double lng2, double lat2) {
- double a = (Math.atan2(lng2 - lng1, lat2 - lat1) * 180d / Math.PI);
- if (a < 0) {
- a += 360;
- }
- return a;
- }
-
- public static final double R=22.5;
- public static final double R_3=R*3;
- public static final double R_5=R*5;
- public static final double R_7=R*7;
- public static final double R_9=R*9;
- public static final double R_11=R*11;
- public static final double R_13=R*13;
- public static final double R_15=R*15;
-
- /**
- * 获得角度方向的空间描
- * 每个象限分为4部分(22.5)
- * @param dirValue 角度方向值
- * @param isFuzzy 是否为模糊描述,当为true时描述信息只表示北、东北、东、东南等八个方向;当
- * 为false时返回以南北为参考的具体角度信息(非西方的以东西为参考),如正北,
- * 北偏东30度等。
- * @return 获得角度方向的空间描述
- */
- public static String getDirectionDesc(int dirValue, boolean isFuzzy) {
- String str1 = "";
- dirValue %= 360;
- if (dirValue < 0) dirValue += 360;
- if (isFuzzy) {
- if (dirValue <= R || dirValue >= R_15) str1 = "北";
- else if (dirValue < R_3) str1 = "东北";
- else if (dirValue <= R_5) str1 = "东";
- else if (dirValue < R_7) str1 = "东南";
- else if (dirValue <= R_9) str1 = "南";
- else if (dirValue < R_11) str1 = "西南";
- else if (dirValue <= R_13) str1 = "西";
- else if (dirValue < R_15) str1 = "西北";
-
- } else {
-
- if(dirValue==0)str1="正北";
- else if(dirValue<90)str1="北偏东";
- else if(dirValue==90) str1="正东";
- else if(dirValue<180 && dirValue>90) str1="南偏东";
- else if(dirValue==180) str1="正南";
- else if(dirValue<270 && dirValue>180) str1="南偏西";
- else if(dirValue==270) str1="正西";
- else if(dirValue<360 && dirValue>270) str1="北偏西";
-
- }
-
-
- return str1;
- }
-
- /**
- * 将double型的数四舍五入成多少位的数,当length大于0时,表示四舍五入小数点后length位;当length等于0时,表示四舍五入
- * 到整数;当length小于0时,表示四舍五入到整数部分前-length位。
- * @param value 待转换的数值
- * @param length 截取长度
- * @return 四舍五入后的数值
- */
- public static String roundTo(double value, int length) {
- value *= Math.pow(10, length);
- long result = Math.round(value);
-
- double newValue = result * Math.pow(10, -length);
- if (length <= 0) {
- return String.valueOf((long) newValue);
- } else {
- String resultStr = String.valueOf(newValue);
- int pos = resultStr.indexOf(".");
- int newLength = resultStr.length() - pos - 1;
- for (int i = newLength; i < length; i++) {
- resultStr += "0";
- }
- return resultStr.substring(0, pos + 1 + length);
- }
- }
-
- public static String readGeoloc(byte[] columnBlob) throws IOException {
- ByteArrayInputStream bais=new ByteArrayInputStream(columnBlob);
- BufferedInputStream b=new BufferedInputStream(new GZIPInputStream(bais));
- BufferedReader br=new BufferedReader(new InputStreamReader(b));
- StringBuffer sb=new StringBuffer();
- String p=br.readLine();
- while(p!=null){
- sb.append(p);
- p=br.readLine();
- }
-
- br.close();
- b.close();
- bais.close();
- p=sb.toString();
- sb.delete(0, p.length());
- return p;
- }
-
-
- }
Wkt类提供了合并wkt数据的方法,代码如下:
- package com.sina.weibo.process;
-
- import com.sina.weibo.entity.WktObject;
- import com.sina.weibo.util.FileUtils;
- import com.sina.weibo.util.GeoUtils;
- import com.vividsolutions.jts.geom.Geometry;
- import com.vividsolutions.jts.geom.MultiPolygon;
-
- import java.util.ArrayList;
- import java.util.HashMap;
- import java.util.List;
- import java.util.Map;
- import java.util.stream.Collectors;
-
- /**
- * @author bingqing5
- * @date 2022/08/03
- * @version 1.0
- */
-
- public class Wkt {
-
-
- /**
- * 合并wkt数据
- *
- * @param list wkt列表
- * @return wkt文本
- */
- public static String mergeWkt(List
list) { - Geometry multiPolygon = null;
- int mc = 0;
- for (int i = 0; i < list.size(); i++) {
- String w = list.get(i).toUpperCase();
- if (w.startsWith("POLYGON") || w.startsWith("MULTIPOLYGON")) {
- MultiPolygon multiPolygonTmp = GeoUtils.getMultiPolygon(w);
- mc++;
- if (multiPolygon == null) {
- multiPolygon = (Geometry) multiPolygonTmp;
- } else {
- multiPolygon = multiPolygon.union(multiPolygonTmp);
- }
- }
- }
-
- return multiPolygon.toText();
- }
- }
该方法在运行时会消耗大量的内存资源,而且数据处理时间较长。这是需要注意的点。