如何删除emacs中的重复行

Question

我有一个很多行的文本，我的问题是如何删除emacs中的重复行？在emacs或elisp软件包中使用该命令，而无需外部utils。

例如：

this is line a 
this is line b 
this is line a 

删除

this is line a 
this is line b 

Answer 1

将这个代码到你的.emacs 3号线（同第1线）：

(defun uniq-lines (beg end) 
    "Unique lines in region. 
Called from a program, there are two arguments: 
BEG and END (region to sort)." 
    (interactive "r") 
    (save-excursion 
    (save-restriction 
     (narrow-to-region beg end) 
     (goto-char (point-min)) 
     (while (not (eobp)) 
     (kill-line 1) 
     (yank) 
     (let ((next-line (point))) 
      (while 
       (re-search-forward 
       (format "^%s" (regexp-quote (car kill-ring))) nil t) 
      (replace-match "" nil nil)) 
      (goto-char next-line)))))) 

用法：

M-x uniq-lines 

Answer 2

(defun unique-lines (start end) 
    "This will remove all duplicating lines in the region. 
Note empty lines count as duplicates of the empy line! All empy lines are 
removed sans the first one, which may be confusing!" 
    (interactive "r") 
    (let ((hash (make-hash-table :test #'equal)) (i -1)) 
    (dolist (s (split-string (buffer-substring-no-properties start end) "$" t) 
       (let ((lines (make-vector (1+ i) nil))) 
       (maphash 
        (lambda (key value) (setf (aref lines value) key)) 
        hash) 
       (kill-region start end) 
       (insert (mapconcat #'identity lines "\n")))) 
     (setq s       ; because Emacs can't properly 
             ; split lines :/ 
      (substring 
      s (position-if 
       (lambda (x) 
        (not (or (char-equal ?\n x) (char-equal ?\r x)))) s))) 
     (unless (gethash s hash) 
     (setf (gethash s hash) (incf i)))))) 

一种替代方案：

• 不会使用撤消记录存储匹配。
• 一般来说会更快（但如果你是在极限速度之后 - 建立一个前缀树）。
• 具有替换所有以前的换行符的效果，无论它们是否与\n（UNIX样式）一致。根据您的情况，这可能是奖金或劣势。
• 如果您重新实现split-string以使其接受字符而不是正则表达式，您可以使它更好一点（更快）。

---

稍长，但是，也许，更多的有效的变体：

(defun split-string-chars (string chars &optional omit-nulls) 
    (let ((separators (make-hash-table)) 
     (last 0) 
     current 
     result) 
    (dolist (c chars) (setf (gethash c separators) t)) 
    (dotimes (i (length string) 
       (progn 
       (when (< last i) 
        (push (substring string last i) result)) 
       (reverse result))) 
     (setq current (aref string i)) 
     (when (gethash current separators) 
     (when (or (and (not omit-nulls) (= (1+ last) i)) 
        (/= last i)) 
      (push (substring string last i) result)) 
     (setq last (1+ i)))))) 

(defun unique-lines (start end) 
    "This will remove all duplicating lines in the region. 
Note empty lines count as duplicates of the empy line! All empy lines are 
removed sans the first one, which may be confusing!" 
    (interactive "r") 
    (let ((hash (make-hash-table :test #'equal)) (i -1)) 
    (dolist (s (split-string-chars 
       (buffer-substring-no-properties start end) '(?\n) t) 
       (let ((lines (make-vector (1+ i) nil))) 
       (maphash 
        (lambda (key value) (setf (aref lines value) key)) 
        hash) 
       (kill-region start end) 
       (insert (mapconcat #'identity lines "\n")))) 
     (unless (gethash s hash) 
     (setf (gethash s hash) (incf i)))))) 

Answer 3

在linux中，选择区域，并键入

M-| uniq <RETURN> 

结果没有重复在新的缓冲区。

Answer 4

如果你有Emacs 24.4或更新的版本，最简单的方法是使用新的delete-duplicate-lines函数。需要注意的是

• 这个工作在一个区域，而不是一个缓冲区，所以选择所需的文本第一
• 它保持原稿的相对顺序，杀死副本

例如，如果你的输入

test 
dup 
dup 
one 
two 
one 
three 
one 
test 
five 

M-x delete-duplicate-lines将使

test 
dup 
one 
two 
three 
five 

您可以选择使用通用参数（C-u）作为前缀搜索。结果将是

dup 
two 
three 
one 
test 
five 

信贷去emacsredux.com。

其他迂回选项，不给完全一样的结果，可以通过ESHELL：

1. sort -u;不保持原件的相对顺序
2. uniq;更糟糕的是，它需要将其输入进行排序