提升Java字符串编码解码性能的技巧(2)
5.快速遍历字符串的办法
无论JDK什么版本,String.charAt都是一个较大的开销,JIT的优化效果并不好,无法消除参数index范围检测的开销,不如直接操作String里面的value数组。
public final class String {
private final char value[];
public char charAt(int index) {
if ((index < 0) || (index >= value.length)) {
throw new StringIndexOutOfBoundsException(index);
}
return value[index];
}
}
在JDK 9之后的版本,charAt开销更大
public final class String {
private final byte[] value;
private final byte coder;
public char charAt(int index) {
if (isLatin1()) {
return StringLatin1.charAt(value, index);
} else {
return StringUTF16.charAt(value, index);
}
}
}
5.1 获取String.value的方法
获取String.value的方法有如下:
·使用Field反射
· 使用Unsafe
Unsafe和Field反射在JDK 8 JMH的比较数据如下:
Benchmark Mode Cnt Score Error Units
StringGetValueBenchmark.reflect thrpt 5 438374.685 ± 1032.028 ops/ms
StringGetValueBenchmark.unsafe thrpt 5 1302654.150 ± 59169.706 ops/ms
5.1.1 使用反射获取String.value
static Field valueField;
static {
try {
valueField = String.class.getDeclaredField("value");
valueField.setAccessible(true);
} catch (NoSuchFieldException ignored) {}
}
////////////////////////////////////////////
char[] chars = (char[]) valueField.get(str);
5.1.2 使用Unsafe获取String.valuestatic
static long valueFieldOffset;
static {
try {
Field valueField = String.class.getDeclaredField("value");
valueFieldOffset = UNSAFE.objectFieldOffset(valueField);
} catch (NoSuchFieldException ignored) {}
}
////////////////////////////////////////////
char[] chars = (char[]) UNSAFE.getObject(str, valueFieldOffset);
static long valueFieldOffset;
static long coderFieldOffset;
static {
try {
Field valueField = String.class.getDeclaredField("value");
valueFieldOffset = UNSAFE.objectFieldOffset(valueField);
Field coderField = String.class.getDeclaredField("coder");
coderFieldOffset = UNSAFE.objectFieldOffset(coderField);
} catch (NoSuchFieldException ignored) {}
}
////////////////////////////////////////////
byte coder = UNSAFE.getObject(str, coderFieldOffset);
byte[] bytes = (byte[]) UNSAFE.getObject(str, valueFieldOffset);
6.更快的encodeUTF8方法
当能直接获取到String.value时,就可以直接对其做encodeUTF8操作,会比String.getBytes(StandardCharsets.UTF_8)性能好很多。
6.1 JDK8高性能encodeUTF8的方法
public static int encodeUTF8(char[] src, int offset, int len, byte[] dst, int dp) {
int sl = offset + len;
int dlASCII = dp + Math.min(len, dst.length);
// ASCII only optimized loop
while (dp < dlASCII && src[offset] < '\u0080') {
dst[dp++] = (byte) src[offset++];
}
while (offset < sl) {
char c = src[offset++];
if (c < 0x80) {
// Have at most seven bits
dst[dp++] = (byte) c;
} else if (c < 0x800) {
// 2 bytes, 11 bits
dst[dp++] = (byte) (0xc0 | (c >> 6));
dst[dp++] = (byte) (0x80 | (c & 0x3f));
} else if (c >= '\uD800' && c < ('\uDFFF' + 1)) { //Character.isSurrogate(c) but 1.7
final int uc;
int ip = offset - 1;
if (c >= '\uD800' && c < ('\uDBFF' + 1)) { // Character.isHighSurrogate(c)
if (sl - ip < 2) {
uc = -1;
} else {
char d = src[ip + 1];
// d >= '\uDC00' && d < ('\uDFFF' + 1)
if (d >= '\uDC00' && d < ('\uDFFF' + 1)) { // Character.isLowSurrogate(d)
uc = ((c << 10) + d) + (0x010000 - ('\uD800' << 10) - '\uDC00'); // Character.toCodePoint(c, d)
} else {
dst[dp++] = (byte) '?';
continue;
}
}
} else {
//
if (c >= '\uDC00' && c < ('\uDFFF' + 1)) { // Character.isLowSurrogate(c)
dst[dp++] = (byte) '?';
continue;
} else {
uc = c;
}
}
if (uc < 0) {
dst[dp++] = (byte) '?';
} else {
dst[dp++] = (byte) (0xf0 | ((uc >> 18)));
dst[dp++] = (byte) (0x80 | ((uc >> 12) & 0x3f));
dst[dp++] = (byte) (0x80 | ((uc >> 6) & 0x3f));
dst[dp++] = (byte) (0x80 | (uc & 0x3f));
offset++; // 2 chars
}
} else {
// 3 bytes, 16 bits
dst[dp++] = (byte) (0xe0 | ((c >> 12)));
dst[dp++] = (byte) (0x80 | ((c >> 6) & 0x3f));
dst[dp++] = (byte) (0x80 | (c & 0x3f));
}
}
return dp;
}
·使用encodeUTF8方法举例
char[] chars = UNSAFE.getObject(str, valueFieldOffset);
// ensureCapacity(chars.length * 3)
byte[] bytes = ...; //
int bytesLength = IOUtils.encodeUTF8(chars, 0, chars.length, bytes, bytesOffset);
这样encodeUTF8操作,不会有多余的arrayCopy操作,性能会得到提升。
6.1.1 性能测试比较
测试代码
public class EncodeUTF8Benchmark {
static String STR = "01234567890ABCDEFGHIJKLMNOPQRSTUVWZYZabcdefghijklmnopqrstuvwzyz一二三四五六七八九十";
static byte[] out;
static long valueFieldOffset;
static {
out = new byte[STR.length() * 3];
try {
Field valueField = String.class.getDeclaredField("value");
valueFieldOffset = UnsafeUtils.UNSAFE.objectFieldOffset(valueField);
} catch (NoSuchFieldException e) {
e.printStackTrace();
}
}
@Benchmark
public void unsafeEncodeUTF8() throws Exception {
char[] chars = (char[]) UnsafeUtils.UNSAFE.getObject(STR, valueFieldOffset);
int len = IOUtils.encodeUTF8(chars, 0, chars.length, out, 0);
}
@Benchmark
public void getBytesUTF8() throws Exception {
byte[] bytes = STR.getBytes(StandardCharsets.UTF_8);
System.arraycopy(bytes, 0, out, 0, bytes.length);
}
public static void main(String[] args) throws RunnerException {
Options options = new OptionsBuilder()
.include(EncodeUTF8Benchmark.class.getName())
.mode(Mode.Throughput)
.timeUnit(TimeUnit.MILLISECONDS)
.forks(1)
.build();
new Runner(options).run();
}
}
测试结果
EncodeUTF8Benchmark.getBytesUTF8 thrpt 5 20690.960 ± 5431.442 ops/ms
EncodeUTF8Benchmark.unsafeEncodeUTF8 thrpt 5 34508.606 ± 55.510 ops/ms
从结果来看,通过unsafe + 直接调用encodeUTF8方法, 编码的所需要开销是newStringUTF8的58%。
6.2 JDK9/11/17高性能encodeUTF8的方法
public static int encodeUTF8(byte[] src, int offset, int len, byte[] dst, int dp) {
int sl = offset + len;
while (offset < sl) {
byte b0 = src[offset++];
byte b1 = src[offset++];
if (b1 == 0 && b0 >= 0) {
dst[dp++] = b0;
} else {
char c = (char)(((b0 & 0xff) << 0) | ((b1 & 0xff) << 8));
if (c < 0x800) {
// 2 bytes, 11 bits
dst[dp++] = (byte) (0xc0 | (c >> 6));
dst[dp++] = (byte) (0x80 | (c & 0x3f));
} else if (c >= '\uD800' && c < ('\uDFFF' + 1)) { //Character.isSurrogate(c) but 1.7
final int uc;
int ip = offset - 1;
if (c >= '\uD800' && c < ('\uDBFF' + 1)) { // Character.isHighSurrogate(c)
if (sl - ip < 2) {
uc = -1;
} else {
b0 = src[ip + 1];
b1 = src[ip + 2];
char d = (char) (((b0 & 0xff) << 0) | ((b1 & 0xff) << 8));
// d >= '\uDC00' && d < ('\uDFFF' + 1)
if (d >= '\uDC00' && d < ('\uDFFF' + 1)) { // Character.isLowSurrogate(d)
uc = ((c << 10) + d) + (0x010000 - ('\uD800' << 10) - '\uDC00'); // Character.toCodePoint(c, d)
} else {
return -1;
}
}
} else {
//
if (c >= '\uDC00' && c < ('\uDFFF' + 1)) { // Character.isLowSurrogate(c)
return -1;
} else {
uc = c;
}
}
if (uc < 0) {
dst[dp++] = (byte) '?';
} else {
dst[dp++] = (byte) (0xf0 | ((uc >> 18)));
dst[dp++] = (byte) (0x80 | ((uc >> 12) & 0x3f));
dst[dp++] = (byte) (0x80 | ((uc >> 6) & 0x3f));
dst[dp++] = (byte) (0x80 | (uc & 0x3f));
offset++; // 2 chars
}
} else {
// 3 bytes, 16 bits
dst[dp++] = (byte) (0xe0 | ((c >> 12)));
dst[dp++] = (byte) (0x80 | ((c >> 6) & 0x3f));
dst[dp++] = (byte) (0x80 | (c & 0x3f));
}
}
}
return dp;
}
使用encodeUTF8方法举例
byte coder = UNSAFE.getObject(str, coderFieldOffset);
byte[] value = UNSAFE.getObject(str, coderFieldOffset);
if (coder == 0) {
// ascii arraycopy
} else {
// ensureCapacity(chars.length * 3)
byte[] bytes = ...; //
int bytesLength = IOUtils.encodeUTF8(value, 0, value.length, bytes, bytesOffset);
}
这样encodeUTF8操作,不会有多余的arrayCopy操作,性能会得到提升。
相关阅读:
- 使用 Python Pip 的十个技巧 (liqianqian1116, 2022-5-12)
- Python 关于字典的操作,看这个就够了 (liqianqian1116, 2022-5-16)
- Java 语言实现简易版扫码登录 (liqianqian1116, 2022-6-24)
- Python中堪称神仙的六个内置函数 (liqianqian1116, 2022-5-13)
- 用 Python 绘制动态可视化图表,太酷了! (liqianqian1116, 2022-5-18)
- 这款 Python 数据可视化工具强的很! (liqianqian1116, 2022-5-20)
- Python这些操作,逆天且实用! (liqianqian1116, 2022-5-17)
- 提升Java字符串编码解码性能的技巧(1) (liqianqian1116, 2022-5-18)
标题搜索
我的存档
数据统计
- 访问量: 231425
- 日志数: 894
- 建立时间: 2020-08-11
- 更新时间: 2024-03-08