What is end-to-end AI search?
Simplicity is critical for search engines, but building one that enables that simplicity is complex. Over the last 20+ years ...
Director of Product Management
Simplicity is critical for search engines, but building one that enables that simplicity is complex. Over the last 20+ years ...
Director of Product Management
Many new AI-powered search solutions have been released this year, and each promises to provide great results, but as ...
Marketing Campaign Production Manager
When you think of “customer experience,” what comes to mind? People, right? Specifically, consumers. Retail customers. That’s easy; the ...
Sr. SEO Web Digital Marketing Manager
A potential customer is about to land on the home page of your ecommerce platform, curious to see what cool ...
Search and Discovery writer
By now, everyone’s had the opportunity to experiment with AI tools like ChatGPT or Midjourney and ponder their inner ...
Director, Product Marketing
Search has been around for a while, to the point that it is now considered a standard requirement in many ...
Senior Machine Learning Engineer
With the advent of artificial intelligence (AI) technologies enabling services such as Alexa, Google search, and self-driving cars, the ...
VP Corporate Marketing
It’s no secret that B2B (business-to-business) transactions have largely migrated online. According to Gartner, by 2025, 80 ...
Sr. SEO Web Digital Marketing Manager
Twice a year, B2B Online brings together industry leaders to discuss the trends affecting the B2B ecommerce industry. At the ...
Director of Product Marketing & Strategy
This is Part 2 of a series that dives into the transformational journey made by digital merchandising to drive positive ...
Benoit Reulier &
Reshma Iyer
Get ready for the ride: online shopping is about to be completely upended by AI. Over the past few years ...
Director, User Experience & UI Platform
Remember life before online shopping? When you had to actually leave the house for a brick-and-mortar store to ...
Search and Discovery writer
If you imagine pushing a virtual shopping cart down the aisles of an online store, or browsing items in an ...
Sr. SEO Web Digital Marketing Manager
Remember the world before the convenience of online commerce? Before the pandemic, before the proliferation of ecommerce sites, when the ...
Search and Discovery writer
Artificial intelligence (AI) is no longer just the stuff of scary futuristic movies; it’s recently burst into the headlines ...
Search and Discovery writer
Imagine you are the CTO of a company that has just undergone a massive decade long digital transformation. You’ve ...
CTO @Algolia
Did you know that the tiny search bar at the top of many ecommerce sites can offer an outsized return ...
Director, Digital Marketing
Artificial intelligence (AI) has quickly moved from hot topic to everyday life. Now, ecommerce businesses are beginning to clearly see ...
VP of Product
Jan 24th 2014 customers
We are Hacker News readers and probably just like you, there is not a day that goes by we don’t use it. It is a little like checking the weather app of the tech world. Long story short, Hacker News is awesome, and we wanted to add our two cents to make it even greater to use.
Indeed, here is our problem: how do we instantly access the old posts we wish we had saved?
Up until now we’ve been using hnsearch.com, maintained for years by the great folks at Octopart. I hope we speak on behalf of the HN community here, we are all grateful for the work they put in hnsearch.com and they inspired us to pursue their effort.
Back in September 2013, we created a “homemade Hacker News crawler” and built a search engine with the data we could get. It was not perfect but somehow, it did the job fine.
Now part of the Ycombinator W14 batch, we have a direct access to the data and it has allowed us to provide instant search for the entire content of Hacker News, 1.2 million articles, 5.2 million comments as of today. See for yourself right here: hn.algolia.com
# Yep, that's a Lisp API :) EXPORT_REGEXP = %r{^((d+) (story|comment|poll|pollopt) "(.+)" (d+) (?:nil|"(.*)") (?:nil|"(.+)") (?:nil|"(.*)") (?:nil|-?(d+)) (?:nil|(([d ]+))) (?:nil|(d+)))$}
(timeout 60 xvfb-run --auto-servernum --server-args="-screen 0, 1024x768x24" wkhtmltoimage-amd64 --height 768 --use-xserver--javascript-delay 30000 "$URL" "$FILE" || timeout 60 xvfb-run --auto-servernum --server-args="-screen 0, 1024x768x24" wkhtmltoimage-amd64 --height 768 --use-xserver --disable-javascript "$URL" "$FILE") && convert "$FILE" -resize '100!x100' "$FILE"
AWS::S3::S3Object.store("#{id}.png", open(temp_file), 'hnsearch', access: :public_read)
class Item < ActiveRecord::Base include AlgoliaSearch algoliasearch per_environment: true do # the list of attributes sent to Algolia's API attribute :created_at, :title, :url, :author, :points, :story_text, :comment_text, :author, :num_comments, :story_id, :story_title, :story_url attribute :created_at_i do created_at.to_i end # The order of the attributes sets their respective importance. # `title` is more important than `{story,comment}_text`, `{story,comment}_text` more than `url`, `url` more than `author` # btw, do not take into account position to avoid first word match boost attributesToIndex ['unordered(title)', 'unordered(story_text)', 'unordered(comment_text)', 'unordered(url)', 'author', 'created_at_i'] # add tags used for filtering tags do [item_type, "author_#{author}", "story_#{story_id}"] end # Custom ranking allows to automatically sort the results by a custom criteria # in this case, a decreasing sort of the number of HN points and comments. customRanking ['desc(points)', 'desc(num_comments)'] # controls the way results are sorted sorting on the following 4 criteria (one after another) # I removed the 'exact' match critera (improve 1-words query relevance, doesn't fit HNSearch needs) ranking ['typo', 'proximity', 'attribute', 'custom'] # google+, $1.5M raises, C#: we love you separatorsToIndex '+#$' end def story_text item_type_cd != Item.comment ? text : nil end def story_title comment? && story ? story.title : nil end def story_url comment? && story ? story.url : nil end def comment_text comment? ? text : nil end def comment? item_type_cd == Item.comment end def num_comments item_type_cd == Item.story ? story_comments.count : nil end end
There is still room for improvement and we would love to know how you are searching for news on HN. What is important for you? Are you searching by date, by upvote, by comment or by user? All together maybe?
We would love to have your feedback! Don’t hesitate to checkout the code: We open-sourced it.
Special thanks to the Octopart and YC teams for making this experience possible!
Give it a try now: hn.algolia.com
Powered by Algolia Recommend