Statistics
| Branch: | Tag: | Revision:

root / lib / filters @ f8d23451

# Date Author Comment
f8d23451 10/06/2011 10:59 am Hiroyuki Yamamoto

sylfilter: added option to specify filtering method.
cd92d070 10/05/2011 04:27 pm Hiroyuki Yamamoto

also use total word number for Robinson-Fisher scale factor.
f29deabd 10/05/2011 03:22 pm Hiroyuki Yamamoto

implemented Robinson-Fisher method to calculate combined probability.
b66a0587 09/22/2011 03:07 pm Hiroyuki Yamamoto

added APIs for global configuration. Added no-bias option.
ef372ee8 09/22/2011 01:33 pm Hiroyuki Yamamoto

add AC_CANONICAL_SYSTEM to configure.ac for target variable.
f10b98d6 09/21/2011 02:31 pm Hiroyuki Yamamoto

added --enable-windows to configure.
e2dc4a59 09/21/2011 01:28 pm Hiroyuki Yamamoto

Add GDBM support.
Add mode switch for standalone and libsylph app.
3c215c37 09/16/2011 03:50 pm Hiroyuki Yamamoto

changed n-gram processing to 4-gram.
a88bc9d0 09/15/2011 04:31 pm Hiroyuki Yamamoto

implemented trigram filter in wordsep-filter.
d8d2c92d 09/13/2011 11:26 am Hiroyuki Yamamoto

made status.dat robust.
63e002d0 09/12/2011 05:58 pm Hiroyuki Yamamoto

removed unrequired freopen().
d75428f4 09/12/2011 05:47 pm Hiroyuki Yamamoto

use plain text file to store status.
fbcd41fa 09/08/2011 05:12 pm Hiroyuki Yamamoto

bayes-filter.c: use transaction for prob.db update.
0b67a65a 09/02/2011 04:55 pm Hiroyuki Yamamoto

speedup learning.
d1c95f4e 09/02/2011 04:18 pm Hiroyuki Yamamoto

speedup classifying.
fca7eb7e 09/01/2011 06:01 pm Hiroyuki Yamamoto

introduced limit of max token length.
fd8e2542 09/01/2011 02:43 pm Hiroyuki Yamamoto

addes SQLite3 support.
aebfd4cc 08/31/2011 04:07 pm Hiroyuki Yamamoto

Made the license BSD-like one.
fb7b6f35 08/31/2011 01:36 pm Hiroyuki Yamamoto

textcontent-filter.c: remove garbage lines (such as base64 data)
4b7e2419 08/30/2011 03:59 pm Hiroyuki Yamamoto

changed probability denominator from message number to word number to lessen the bias.
9b245a85 08/30/2011 03:30 pm Hiroyuki Yamamoto

extract all text parts and filenames.
da8a1280 08/30/2011 11:44 am Hiroyuki Yamamoto

weighted probability for small frequency number of words.
6fb72384 08/30/2011 10:57 am Hiroyuki Yamamoto

check for QDBM on configure.
a0b81e12 08/29/2011 05:24 pm Hiroyuki Yamamoto

implemented unregister of messages.
6177032a 08/29/2011 04:48 pm Hiroyuki Yamamoto

link libsylph-builtin to libsylfilter.
519f8cf9 08/29/2011 03:26 pm Hiroyuki Yamamoto

use fixed db path under ~/.sylfilter .
fc4bd9f7 08/29/2011 02:59 pm Hiroyuki Yamamoto

add parameter for directory path to xfilter_bayes_db_init().
cfadedec 08/29/2011 11:55 am Hiroyuki Yamamoto

lib/filters/bayes-filter.c: skip if either junk or clean database is
empty.
src/sylfilter.c: fixed bulk learning.
704a6ae7 08/25/2011 06:11 pm Hiroyuki Yamamoto

enabled multiple file testing. Fixed error output.
0113a62c 08/25/2011 04:49 pm Hiroyuki Yamamoto

implemented debug mode switch.
ce81a473 08/25/2011 03:47 pm Hiroyuki Yamamoto

handle URL in word separator.
fe4d11cc 08/25/2011 01:44 pm Hiroyuki Yamamoto

modified word-separator.
8d7dcace 08/25/2011 01:14 pm Hiroyuki Yamamoto

added built-in libsylph, and use it from textcontent-filter.c.
3815f7e2 08/24/2011 05:43 pm Hiroyuki Yamamoto

commented out debug output (wordsep-filter.c)
940d391b 08/24/2011 05:38 pm Hiroyuki Yamamoto

break after full-width alphabets.
3e93864c 08/24/2011 04:34 pm Hiroyuki Yamamoto

removed unused code in bayes-filter.c.
5c9fbf02 08/24/2011 03:57 pm Hiroyuki Yamamoto

tuned probs of clean-only words.
f1db2e5b 08/24/2011 03:47 pm Hiroyuki Yamamoto

fixed word separator.
34ae426c 08/24/2011 03:20 pm Hiroyuki Yamamoto

implemented word degeneration.
54aea252 08/23/2011 06:01 pm Hiroyuki Yamamoto

also parse specific headers.
93ecf147 08/22/2011 05:17 pm Hiroyuki Yamamoto

add bias for junk-only tokens.
e260dbfd 08/22/2011 05:04 pm Hiroyuki Yamamoto

sylfilter: allow multiple file input.
fd3b9e72 08/22/2011 04:39 pm Hiroyuki Yamamoto

treat some special characters word-letters.
afc49e81 08/22/2011 03:24 pm Hiroyuki Yamamoto

drop two-letters hiragana.
15b78962 08/22/2011 03:06 pm Hiroyuki Yamamoto

modified debug output.
af7ecefa 08/22/2011 02:01 pm Hiroyuki Yamamoto

added option to show learning status.
4cab1247 08/22/2011 01:03 pm Hiroyuki Yamamoto

renamed old sylfilter.c to sylfilter-test.c and added new sylfilter.c.
faa55f07 08/18/2011 06:30 pm Hiroyuki Yamamoto

implemented basic bayesian filter.
28703c7f 08/18/2011 02:06 pm Hiroyuki Yamamoto

modified interface of XFilter result. Added API to open/close bayes db.
29dd8dfb 08/17/2011 04:43 pm Hiroyuki Yamamoto

added API to get kvs total sum.
b45b0705 08/17/2011 04:13 pm Hiroyuki Yamamoto

added API to get kvs record number.
761f2955 08/17/2011 03:20 pm Hiroyuki Yamamoto

implemented KVS APIs and QDBM implementation.
ebcfee46 08/17/2011 12:53 pm Hiroyuki Yamamoto

don't normalize case.
cbb0d1ac 08/16/2011 05:06 pm Hiroyuki Yamamoto

drop OTHER_SYMBOL on word separator.
de7bb84e 08/16/2011 04:23 pm Hiroyuki Yamamoto

bayes-filter.c: added word-count func.
5a583287 08/16/2011 02:37 pm Hiroyuki Yamamoto

implemented simple stopword.
ccd4115d 08/16/2011 11:17 am Hiroyuki Yamamoto

Implemented simple word separator.
4012ec30 08/09/2011 04:05 pm Hiroyuki Yamamoto

Initial commit