Jedis（Redis）减速

Question

我将很多文本插入到redis中以便逐行存储频率。但是，jedis/redis在执行一定操作后需要花费大量时间执行 e操作，并且程序以错误结束： java.lang.OutOfMemoryError。

这里是我的测试主要文件： 公共类临时{

private ExecutorService executor; 

public temp() { 
    executor = Executors.newFixedThreadPool(5); 
} 

public static void main(String[] args) { 
    temp ob = new temp(); 

    System.out.println("starting"); 
    for(long i =0;i<10000000;i++) { 
     if (i%10000 == 0) { 
      System.out.println(i); 
     } 
     String x = Integer.toString(new Random().nextInt()); 
     ob.executor.submit(new Runner1("abra"+x)); 
     ob.executor.submit(new Runner2("delhi"+x)); 
    } 

    try { 
     if (ob.executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS)) { 
      System.out.println("completed"); 
     } 
    } catch (Exception ex) { 
     ex.printStackTrace(); 
    } 

} 


} 

这里是我的两个参赛者： 亚军1：

public class Runner1 implements Runnable { 
//private static RedisClientUtil redisClient = null; 
private String key; 
private static Integer count = 0; 

public Runner1(String key) { 
    this.key = key; 
} 

public void run() { 
    try { 

     ArrayList<ArrayList<Object>> cmd = new ArrayList<ArrayList<Object>>(); 

     String offer_title = this.key + " this is thread1"; 
     String offer_title_words[] = offer_title.split(" "); 
     for (String word : offer_title_words) { 
      // INCR the frequency in reddis 
      cmd.add(GenerateUtils.getArrayList("incrBy", "test"+word, 1)); 
     } 

     List<Object> responses = RedisbenchmarkTest.getLocalhostJedisPool().executePipelinedAndReturnResponses(0,cmd); 
     cmd = null; 
     responses = null; 

     updateNumberOfRowsInserted(); 

    } catch (Exception ex) { 
     ex.printStackTrace(); 
    } 
} 

    private synchronized void updateNumberOfRowsInserted() { 
     //logging 
     count++; 
     if(count%10000==0) 
      System.out.println("Thread 1 : " + count); 
    } 



} 

亚军2：

public class Runner2 implements Runnable { 
//private static RedisClientUtil redisClient = null; 
private String key; 
private static Integer count = 0; 

public Runner2(String key) { 
    this.key = key; 
} 

public void run() { 
    try { 

     ArrayList<ArrayList<Object>> cmd = new ArrayList<ArrayList<Object>>(); 

     String offer_title = this.key + " this is thread2"; 
     String offer_title_words [] = offer_title.split(" "); 
     for (String word : offer_title_words) { 
      // INCR the category_word in reddis 
      cmd.add(GenerateUtils.getArrayList("incrBy","test1"+word,1)); 
     }     RedisbenchmarkTest.getLocalhostJedisPool().executePipelinedWithoutReturningResponses(0,cmd); 
     updateNumberOfRowsInserted(); 

    } catch (Exception ex) { 
     ex.printStackTrace(); 
    } 
} 

private synchronized void updateNumberOfRowsInserted() { 
    //logging 
    count++; 
    if(count%10000==0) 
     System.out.println("Thread 2 : " + count); 
} 

} 

这是我的Redis客户端：

public class RedisbenchmarkTest { 

    private static RedisClientUtil localhostJedisPool; 


private static final JedisPoolConfig standardJedisPoolConfig = new JedisPoolConfig() {{ 
    setMaxTotal(500); 
    setMaxIdle(20); 
    setMaxWaitMillis(500); 
    setTestOnBorrow(false); 
    setTestOnReturn(false); 
}}; 

private static final int semiLowTimeout = 500; 

static { 
    initialize(); 
} 

public static void initialize() { 
    localhostJedisPool = new RedisClientUtil(
      standardJedisPoolConfig 
      , "localhost" 
      , 6379 
      , semiLowTimeout 
    ); 
    } 



    public static RedisClientUtil getLocalhostJedisPool() { 
     if (localhostJedisPool == null) { 
      initialize(); 
    } 
    return localhostJedisPool; 
    } 
    } 

Answer 1

• 你应该redis.conf禁用bgsave如果Redis的开始bgsave通常巨大滞后暗示，直到保存完成。
• 您应该确保redis.conf中的maxmemory足够高。我猜你的实例正在运行。

Answer 2

Redis不是内存不足的那个，你的Java过程是。

您没有发布完整的实际错误，我不是一个Java的人，但Caused by: java.lang.OutOfMemoryError: Java heap space可能会为您提供一些答案。