I marked some files as junk which were known to me to be junk files viruses & spam. The strange thing is after I did this when I did searches for other files I had files showing up as junk files that were in no way associated with the files I had selected as junk files.

The files in question were known to me to be good files. I did a bitzi on them and they were known good files in bitzi. They were in no way associated with the files I had selected as junk not in name or size.

So why is the junk file filter picking them as junk files ?.

Also can change from junk to not junk and the next search it will show up as junk again.

4.9.39

__________________










A little common sense goes a long way

Later Grandpa

I haven't used it as yet, but you can undo your junk settings in Tools>Options. The junk settings seemed to work for me, but I haven't used it extensively (wind on a mac).

select the files that are not junk and click "Not Junk". The filter has to be trained to you own preferences.

I have done that

__________________










A little common sense goes a long way

Later Grandpa

hmm, could be a bug

Maybe, maybe not it just might take it a while to remember what you want a couple of them quit showing up as junk after the 4th time telling it, it was not junk. But others are still showing up as junk. The ones that are still showing up as junk are pretty close in size as the ones I named as junk. What format are they using for the filter name or file size.

__________________










A little common sense goes a long way

Later Grandpa

Whoops spoke too soon it is back to displaying them as junk. Also the options of show junk at bottom and don't show junk do not appear to work.

From general deduction I am guessing it is filtering by IP address.

__________________










A little common sense goes a long way

Later Grandpa

This is probably a bug related to one I discussed with LW devs previously. I will notify a developer about it.

Thanks for a confirmation on it.

Grandpa have you adjusted the Junk Threshold? I notice LW 4.9.39 now lists this option under Tools>Options>Searching>Junk as compared to 4.9.37 which listed this slider under the search options on the main interface.

Though I must also confirm Grandpa's findings. When trying to filter out virii, & having set it Not to show Junk, it still showed up (but with the trash can beside them.)

Attached Images

junk-virii.gif (1.9 KB, 114 views)

I've had to turn off the junk filter in order to get any "What's New" results.

With my current set-up and habits, the filter was more of a hassle, but quite interesting to test, and working around the spam was already habitual

Other notes about the 4.9.39 Beta:

--About box needs updating (i.e, Dave Nicponski; Roger Kapsi)

-- command-clicking (OS X) on one of several selected entries should deselect the entry. Instead, it deselects everything else and selects the entry (this is old, like the false drag and drop problem in the downloads panel).

--search results are sorting by case in the name column

--when running as UP and with the filter on, heavy CPU activity somehow was triggering a loss of leaves.

--would be nice if the new look for the search tabs applied also to the pane tabs, status bar icons, scroll buttons, and the like.

Yes I played with the slide bar but my findings was it didn't make much difference. What I did when training was I did software searches that had large amounts of the 851.7 virus then I sorted by file size I then chose the whole group of 851.7 files which was apparently a mistake.

I still haven't figured out a way to make it work without it filtering out a large amount of good files. I will keep trying but i have a feeling that this filter is no better than any I have seen in the past and eventually I will give up trying to make it work and totally disable it.

__________________










A little common sense goes a long way

Later Grandpa

For me it worked well with 4.9.37 re: auto-spam. But I installed 4.9.39 before I did any more. I then concentrated upon the virii sizes as you were. I found it wasn't as successful. I don't know whether one can conclude that it works well with auto-spam such as mp3 & ipod spam, but not so well with virii.

* Some notes about how to use the spam filter to its greatest efficiency would be beneficial.

Thanks for the feedback.

In 4.9.40, Roger fixed some display confusion to make the threshold for showing a trash can in the "quality" column equal to the threshold for hiding junk (if the hiding junk option is enabled).

These problems with too many files being marked junk over time are likely a problem with the filter being set up to learn more from bad hints than from good hints. (In the code, these hints are called "tokens".) Hopefully LW 4.9.40 is much better about this; the learning should be much less biased toward the bad unless you set the sensitivity above 50%.

The spam filter is actually a set of filters, where the file starts out being 100% good, and each filter multiplies the goodness by some value between 1.0 (inclusive) and 0.0 (exclusive). It's probably a host of different filters that are whittling the files down to a "junk" rating.

Basically, if you search for some terms and end up getting a result that you mark spam, LW will internally create a bunch of tokens for different things LW knows about the file. There's a token for the size of the file, a token for each word in the title, etc, etc. Tokens that keep showing up in the search results for "very bad" spam rated files gradually get marked more and more "bad". Tokens that keep showing up in the search results for "very good" spam rated files gradually get marked more and more "good". Part of the problem is probaby that the standard for "very good" was more tough than the standard for "very bad", (hard-coded to below 15% junk vs. above 70% junk) so with each search, the effects of the "bad" tokens relative to the "good" tokens was multiplied. Basically, lots of very "bad" tokens mean lots of search results get very bad spam ratings, which means lots of tokens slowly get marked more "bad"... it's a snowball effect, and we need an opposing "good" snowball effect to cancel it out. This is an over-simplification, but hopefully it helps you get a general idea of what goes on inside the spam filter.

Give 4.9.40 a try and let us know how it works for you.

Don't be shy about going into the options and changing the sensitivity of the junk filter. In 4.9.40 (unlike 4.9.39 and 38), to some extent the sensitivity of the junk filter affects the balance of influence between bad junk ratings and good junk ratings. Below a sensitivity of 50%, it's hard to say which way the learnig is biased. Above 50% sensitivity, the learnig becomes more and more biased toward increasing the "bad-ness" of tokens. Hopefully with feedback from real-world useage, we can tweak the filter to have very little bias in the sensitivity ranges that people actually use.

Thanks for the explanation.

Now I am going to have to eat my words the filter in .40 seems to work extremely well so far. I have never liked filters none of them ever seemed to work but so far this one does. It is very easy to train and very effective. I do like the fact that you can view the junk results so if it makes a mistake you can correct it.

Time will tell if it is going to be good or not but so far it is very good you guys once again have proved to me that you are among the best.

Now that that is out of the way how about working on direct connect. You know in the future that may be all that we have.

Any way keep it up you are the King of the hill and everybody is going to try and push you off. But you keep making improvements like you have the last few years, then they are going to have a hell of a time doing it.

__________________










A little common sense goes a long way

Later Grandpa

One thing I've noticed about the win 4.9.40 is the stop search button sometimes seems to become faded out so I can only stop a search by right-clicking the search tab. This has been happening in spurts/groups of searches. Actually I noted this seems to happen after I right-click the tab to repeat the search. Or is this normal. I don't always want to stop a search.

As far as the filter goes, I find that the result still seems to display despite being set not to (I'm concentrating on virus sizes for program files; ie: 765/851 KB.) One moment they'll appear as normal, a few secs later with a trash can as it recognises them as junk. At present I have settings at around 85-90% for the slider. I've increased it bit by bit over this session.

Sometimes a new one shows up in the results window, I click junk & the others with a trash icon disappear as well as that one. I'm meaning only a small % of those detected as spam showed up in the results window. Is this normal. And why do they suddenly disappear only sometimes when another one is designated as junk?