Repository navigation
Weird recommendation behavior #91
Description
When I dislike a course, it recommends it to similar users.
class user < ActiveRecord::Base
has_many :courses
recommends :course
end
user1.like(Course.find(2622))
user1.like(Course.find(2431))
user1.like(Course.find(1466))
user1.dislike(Course.find(315))
user2.like(Course.find(2622))
user2.like(Course.find(2431)
user2.recommended_courses.map{ |c| c.id }
=> [315, 1466]
user2.liked_course_ids
=> [2431, 2622]
user1.liked_course_ids
=> [1466, 2431, 2622]
user1.disliked_course_ids
=> [315]
Activity
davidcelis commentedon Apr 24, 2014
Ah, yeah. The reason for this is that all recommendation values (or the top N values if you've used
config.nearest_neighbors) get stored in Redis. So if you've only been compared with a few other users, it's possible that your top recommendations would include a recommendation with a score of less than 0 (which happens when similar users dislike that item). I wonder if it would be best to exclude any item with a negative score from returned recommendations?hoguej commentedon Apr 24, 2014
Should at least be sorted by highest score, right? 315 should have a negative score, and 1466 is positive. You'd expect 1466 to be first?
hoguej commentedon Apr 24, 2014
But would also make sense to remove negative scores from recommended_(). maybe expose that through a new method not_recommended_ or something.
davidcelis commentedon Apr 24, 2014
It's definitely sorted by highest score. Are you sure that 315 has a negative score, or a lower score than 1466? I'd inspect the Redis ZSET directly. Are there other ratings at work here?
hoguej commentedon Apr 25, 2014
recommendable:users:134255:recommended_courses
value score
315 -1
1466 1
hoguej commentedon Apr 25, 2014
I'm not sure what you mean by ratings, exactly.
Checking redis, it seems like the data is consistent with what I'd expect from the steps I followed in the first post.
hoguej commentedon Apr 25, 2014
So, my course class has a rated? method, unrelated to recommendable. Wonder if that's causing a problem.
hoguej commentedon Apr 25, 2014
Rename that method and reran everything. Same result. I guess next step is to create a minimal example app and try to reproduce the problem there.
davidcelis commentedon Apr 25, 2014
Sorry, typoed that. I meant to ask if there are any other ratings (likes/dislikes) at play here.
davidcelis commentedon Apr 25, 2014
Anyway, please do try to reproduce it in a codebase I can clone down and take a look at. Thanks!
hoguej commentedon Apr 25, 2014
I've created a simple sample app, but noticed something that is probably the culprit.
Notice, that we look for (5,3,4) but we get back [3,4,5]. This is rails 3. Not sure if rails 4 handles this more appropriately.
1.9.3-p484 :001 > user2 = User.find(2); user2.recommended_courses.map{ |c| c.id }
User Load (2.9ms) SELECT "users".* FROM "users" WHERE "users"."id" = ? LIMIT 1 [["id", 2]]
Course Load (0.1ms) SELECT "courses".* FROM "courses" WHERE "courses"."id" IN (5, 3, 4)
=> [3, 4, 5]
1.9.3-p484 :002 >
hoguej commentedon Apr 25, 2014
I do something like this to deal with this same situation from solr. Not the most efficient, but database agnostic. If I could access the raw scores, I could do something like that myself.
courses.sort!{ |x,y| course_ids.index(x.id.to_s) <=> course_ids.index(y.id.to_s) }
davidcelis commentedon Apr 25, 2014
Shoot, you're right: https://github.com/davidcelis/recommendable/blob/master/lib/recommendable/rater/recommender.rb#L21-L27
I thought that I was correctly sorting the records before returning them to the user, but that's not the case. So while that list does contain the best recommendations, they're not sorted within that list. I probably did this originally to keep that query chainable, but I'm willing to wager that it's more important for that list to be in the right order.
hoguej commentedon Apr 25, 2014
Also, seems weird it would even be in the return result, since there's no way to know where the positive stops and the negative starts. You could argue, with a fully populated recommendation engine, it would probably give the correct result. Might be true where you are getting 10 results back, but false if getting back 100's of recommendations.
I can take a crack at a patch, but I'm new to this lib and redis.
davidcelis commentedon Apr 25, 2014
Good point. I think that at the very least, items with negative scores should never be recommended. I can definitely fix that much, but I'll need to think a bit more on sacrificing the chainability of querying against those recommendations for having them ordered correctly. There is a tradeoff there, but I'm leaning towards ordering them correctly.
hoguej commentedon Apr 25, 2014
Wonder if we can dip into the ActiveRecord class to lazily store the order.
On Fri, Apr 25, 2014 at 3:26 PM, David Celis notifications@github.comwrote:
davidcelis commentedon May 18, 2014
@hoguej I decided to abandon leaving the
recommended_thingsmethods chainable so that recommendations are sorted correctly. This has landed as of a0e77bd. I'll leave this open if you want to confirm that it works.hoguej commentedon May 19, 2014
I'll take a look this week. Thanks for the update. When I get to it, I'll expose my test app.
hoguej commentedon May 27, 2014
Sorry, got ill last week. Still on my agenda.