Sidedeal Math

@shawn thanks! That’s not obvious how to get to on mobile, but good to know.

@shawn I’m not sure how machine learning helps here? What’s the benefit over an average?

@DoctorOW how much time do you have? The short answer is it helps compare one product’s rating with other product ratings given the set of products don’t have the same number of reviews.

In other words: take two products.

• Product A: 1 rating of 5. Average rating of 5.
• Product B: 50 ratings. Average rating of 4.5

Which is the better product?

We’d love to just tell you these Stroopwafels are 5-stars but our data tells us to expect some less-than-5 star reviews. These things can’t be that good.

@shawn Regarding your example, it’s hard to tell which is better but I would like to see all the raw data so I could read the reviews. The final rating shouldn’t be anything other than 5 stars if 100% of the people rated the product 5 stars regardless if there’s only 1 or 10,000 ratings. Let me see how many of each rating there is and I’ll decide for myself how strong that 5-star rating is.

@DoctorOW @shawn And you won’t even see my zero star review for stroopwafels since I hate them and didn’t buy them so won’t rate them. Machine learning will never include that.

@cengland0 but if they provide pure data they can’t add to the the algorithm which product they rather push (you see, there are a lot of variables). Just like Amazon promoting stuff without a sponsored tag.

@cengland0 @mwgm All the raw/pure data is available if you want it. Just click the “See All Reviews” button:

@cengland0 @shawn and when a product has 100 reviews is the customer expected to go over them one by one and do the math??
Not to mention you have to click the question mark to get the info that this isn’t customer ratings, it is some ranking based on customer reviews and whatever you added to the algorithm. Presenting this as rating is misleading…
But you’re only doing your job @shawn, that’s ok. I was pointing out the obvious for others, not in order to start a discussion.

@mwgm I agree with you 100% that it is misleading and now my confidence level has gone down which is the opposite of what @shawn was trying to do. Now I know not to trust the “fudged” rating on Sidedeal in the future. Thanks for pointing this out.

@mwgm I don’t expect customers to do any math unless they want to.

@cengland0 what’s an example of a rating system on the internet that you trust?

@shawn I trust eBay, AliExpress, Best Buy, Newegg, and even Amazon. I can look at negative ratings and decide for myself if it is justified.

For example, Amazon customers will rate a product as 1 star when the shipping took a day too long and there’s nothing wrong with the product. At least I have the ability to filter by rating and decide for myself if a lower rating is warranted and I don’t have to look at negative reviews if there is 100% 5-star ratings. But now that I see 4.5 average (or whatever you are calling it) I will seek out those few ratings to see why it’s not 5.0. In the OP example, there is none. That’s what is wrong and misleading.

By the way, I hardly ever read positive reviews and focus more on the negative ones.

@cengland0

@shawn Ok but that shows that they may exclude some ratings based on age or other factors. Do they ever lower the rating to 4.5 when 100% of the reviews are 5 star? I cannot think of any valid reason that would ever happen even if you remove ratings from non verified purchases and old ratings because you are still left with 100% 5-star which should be reflected as 5.0 in the final value displayed to the customer.

@cengland0 Yes. The valid reason is as I said before: so you can better compare products given the set of products don’t have the same number of reviews.

My larger point is, this problem is so well researched and understood that most rating systems (Amazon, IMDb, Reddit, Yelp, Digg, etc) employee techniques like Wilson score confidence intervals, Bernoulli/Binomial distributions, and Bayesian averages to calculate ratings. Here’s a great post from over a decade ago: https://www.evanmiller.org/how-not-to-sort-by-average-rating.html

@shawn I read the page you linked and respectfully disagree with you and that page.

From the page:

It says amazon is wong but they are probably the most successful company in the history of the world with an estimated value of 1.6 Trillion dollars. They are obviously doing something right.

Amazon’s rating system is well known and pretty standard and straightforward. Decreasing an average 5.0 to 4.5 because of a low number of responses is absolutely the wrong thing to do, in my opinion.

I’m just a customer and you can run your site the way you want but I will not trust your product ratings ever again after your explanation today.

@cengland0 Amazon moved to a machine learning model after this blog post in 2009, so you’re unfortunately going to have to not trust Amazon product ratings as well. It is not a simple average.

@shawn from what I read, Amazon will exclude some ratings. That does not mean a product with 100% 5-star ratings will ever show a 4.5 to the users. If you think it does, I would love to see an example.

@cengland0

@shawn Nope, that doesn’t show 100% 5-star and then a lower 4.5 average.

The example I want doesn’t have to be 100% 5-star or a total 4.5, but I want to see a product with 100% one rating and then a final rating that is something different than the one rating.

The sample you provided had 3 different ratings and it’s difficult to figure out which ones were excluded due to age or not a validated purchase. I can assume both 5-stars were invalidated leaving a 1-star and a 3-star so there is a remaining average of 2.0 out of 5.0. There is no way to do the same math on your Sidedeal site that OP posted.

@cengland0 @shawn Product is perfect [one star]! It was a gift, the recipient seemed to like it [five stars].

@cengland0 read the timeline from TommyTripod there. If the first 5-star review was excluded the rating after 3 reviews would have been (1+3)/2=2 to begin with.

The difference between Amazon’s product rating and the simple average rating cannot be explained by simple inclusion and exclusion. We know this because in 2015 they described their algorithm as some sort of weighted average where they “give more weight to newer reviews, reviews from verified Amazon purchasers and those that more customers vote up as being helpful.” You cannot reverse the math because you do not know the weights of the inputs given the data you have available.

@shawn OK; you win! My first Google search landed me here and my brain blew a gasket.

“[W]ell researched and understood” indeed!

Thank you for the peek behind the curtain.

@shawn @TrophyHusband I just read that link. I am truly glad all I have to teach in beginning stats classes is the one mentioned there that is used when flipping coins.

@shawn Exactly, you expect them to not do the math and trust your ranking which includes profit margins and other variables instead of giving a presentable summary of the raw data.
The purpose of this place (and its sister websites) is to sell products, it’s a business which tries to maximize profit. I often think of this forum in a way similar to complemantary snacks some places have, the longer the customers stay the more likley it is they’ll buy and spend money. Adding this ranking system is just another brick in your marketing strategy and that’s ok, as long as you call it ranking, not rating which is very misleading.

@cengland0 Don’t trust Amazon. Anyone can write a review, not just verified purchases, and it’s well known Amazon’s ratings can be misleading. Amazon has many ways of tweaking those numbers within the framework they established.
You can try checking the product on www.fakespot.com
simply enter the product URL and it’ll tell you how fake it thinks the ratings are. I know it’s not accurate but it’s the best we’ve got (that I know of), other than reading the reviews one by one.

Here’s an example (check the adjusted rating and read the overview section):
https://www.fakespot.com/product/bubba-brands-20085-hero-fresh-insulated-stainless-steel-travel-mug-with-grip-24-oz-black

@cengland0 @mwgm @shawn @kidsandliz
while I can understand the issue with not being able to replicate the math, and the concept of AI ‘prediction’ of rating, I am less disappointed in this example since it obviously downgraded the rating, in the anticipation (?) that some folks would not care for it (for instance @whisper3600)

What a waste of money and resources. I would not even give them to the food bank. Made my wife sick. Felt like a ton of lead in her stomach.

@chienfou you assume it anticipated that some folks would not care for it, but you don’t know why it was downvoted as this is not open source. There are many variables, as was pointed out earlier. Perhaps the profit margin isn’t as good as on other products?
Raw data is very different than manipulated data, which is what this is.
The manipulator is not objective, there’s a profit to be made. It is, after all, a business that works on revenue and has to pay employees etc. etc.

@mwgm
Ok I’ll buy that. I guess I assume that meh has altruistic reasons for their rating process because I generally like the site and feel they are pretty trustworthy.
Data manipulation (for retail or in general) is not something I support as a rule. Though if you let me know you are ‘tweaking’ the data for some legitimate (or transparent) reason I might let it ride.

@mwgm profit margin is not a component of our ratings.

@chienfou no retail is altruistic, let’s be honest. It’s not a non-profit. To quote the octopus (though probably not verbatim)- if you can’t buy you shouldn’t be hanging out in a shopping website’s forum.

I agree, I hate data manipulation and almost always try to use open-source (and will donate if I use regularly). Unless the code is open/transparent you cannot know what kind of manipulation is taken place and therefore for something as simple as customer ratings raw data is the way to go, especially when the manipulator has a stake in it.
JMO

@shawn you don’t expect me to go over reasons you might want to push one product over another one by one, do you? Warehouse space, promotionals just to name the first two that come to mind.

@mwgm not at all, but I wanted to point out that profit margin is not a component of our ratings because you stated “your ranking which includes profit margins and other variables” which is inaccurate.

@shawn I stand corrected, I should have worded it better. My bad. Clearly, as I have no knowledge of what goes into the algorithm I cannot make such statements, but the principal stands.

@RiotDemon or you can find Screenshot when using gesture navigation on your phone through the long screen press that you use to switch apps. If you use 2 button nav, yeah, you have to 2 button screenshot now. I am relearning my navigation using Gesture now. Ugh.

@Pamtha ah, I don’t use gesture. Not sure if the 3 has the option.

@RiotDemon I have a 3a XL. Check Settings - System - Gestures - System Navigation to see if it is available. It’s not gestures like hovering over the screen and waving, it’s all swiping and pressing. It’s weird at first, but then it becomes intuitive.

@Pamtha ah yeah. I have the option. I use two button navigation.

@Pamtha trying gestures now to see how it goes.

@Pamtha @RiotDemon
/giphy trying gestures

@ybmuG
pretty sure those gestures will confuse the shit out of the phone…

@Pamtha @RiotDemon @ybmuG Can anyone explain why tech people feel such a need to fix things that work? Is it tied in to why WallyWorld feels a need to move everything around periodically, to make you spend more time? 'Cause the bottom line for the Usr is just more frustration…

@2many2no

Can anyone explain why tech people feel such a need to fix things that work

Tech changes DO seem designed to frustrate the user sometimes (coming from an AARP member and someone qualified for Medicare/Social Security). Incremental changes lead to bigger changes over time. (For instance… remember the old slider phone… that was replaced by touch screens).
As for WallyWorld (and even more specifically Sam’s/Costco) things often get moved around to make you linger in the store longer in the hopes you will pick up a few items not on your shopping list.

@2many2no @Pamtha @RiotDemon @ybmuG
A great example of tech going all in on the wrong horse is Windows 10. Microsoft used to be pretty good at productivity software, but they decided that going to liteweight apps and an interface designed less for office/work use and more focused to be social media friendly was a good idea. IMHO They were wrong

@chienfou @Pamtha @RiotDemon @ybmuG Amen, brother. Win10 does everything Win7 did, but they moved it all around and hid it. Are they mentally transfixed by the search button?

@2many2no
Again an excellent example of stuff they ‘dumbed down’ as they moved away from productivity. Why is it so hard to find the ‘my computer’ (now known as the “this PC”) button if you want to find a file location or check on the hierarchy of a set of folders? Yep… search box to the rescue (because mouse clicks are too complicated…??)

@Pamtha @RiotDemon power and volume down does not work on my 3. I’m relearning gestures now too. I miss two button but I need screenshots often enough to have to change.

@2many2no the consistent inconsistency of windows 10 start menu search amazes me. It may be worse than Amazon’s search. And the press to load every computer with candy crush and other bullshit is just throwing “we’re not really for work” in the face of businesses small enough to not invest in Enterprise edition. Yes, I can tweak around it, but I shouldn’t have to (or I should be able to set one thing in group policy to disable it).

@djslack @Pamtha So I wasn’t a fan of gestures. I hate the way you go backwards… But, even in gestures, power and volume down works for screenshots.

@djslack @Pamtha @RiotDemon Why tech people “fixing” things that aren’t broke? I’d hazard a guess part of it is so they can look like they are busy and the company still needs to keep them on the payroll.

@Pamtha @RiotDemon I finally got it working. I found out you have to hit both at exactly the same time AND hold them for longer than expected. Sucks that it’s no longer a one handed operation. I still wind up turning down the volume some but am glad to be able to go back to 2 button navigation.

@djslack if I try really hard, I can do it one handed, but it’s a hassle. Not sure if the case or the phone’s fault, or the fact that my hands aren’t giant.

@2many2no @Pamtha @RiotDemon @ybmuG
Just saw that the Windows 10 new update is doing away with the system property page…