Fedora Search Engine

[Paweł Kłos]

Hi,

The idea of Fedora Search engine has came back with discussion about 
https://fedorahosted.org/fedora-infrastructure/ticket/1055 ticket on the 
meeting.

Many people was truly interested in participating in that project.

So, if somebody is still interested I propose to make some brainstorm 
about that matter, so we could reach better results than years ago :)

First  we can discuss in that mail thread and move then to irc or some 
other place.

Regards,
Pawel Klos
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Vasyl Kaigorodov]

Hi folks,

I did follow the discussion during the meeting, and was hesitating to ask: why exactly do
we want own search implemented?
From my experience, Google or DuckDuck do index Fedora resources just fine, and whenever I
want to find something - I just use those.
Own search engine adds to the cost of support and maintenance, with questionable
benefits.
Unless we have some public resources that have tricky robots.txt and are not indexed by
other search engines - this looks like a waste of resources to me.
I might miss something though, so any insight is welcome.

Thanks.
--
vk

----- Original Message -----
From: "Paweł Kłos" <pawel(a)pawelklos.eu&gt;
To: "Fedora Infrastructure" <infrastructure(a)lists.fedoraproject.org&gt;
Sent: Thursday, August 27, 2015 9:15:09 PM
Subject: Fedora Search Engine

Hi,

The idea of Fedora Search engine has came back with discussion about 
https://fedorahosted.org/fedora-infrastructure/ticket/1055 ticket on the 
meeting.

Many people was truly interested in participating in that project.

So, if somebody is still interested I propose to make some brainstorm 
about that matter, so we could reach better results than years ago :)

First  we can discuss in that mail thread and move then to irc or some 
other place.

Regards,
Pawel Klos
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Vasyl Kaigorodov]

Hi folks,

I did follow the discussion during the meeting, and was hesitating to ask: why exactly do
we want own search implemented?
From my experience, Google or DuckDuck do index Fedora resources just fine, and whenever I
want to find something - I just use those.
Own search engine adds to the cost of support and maintenance, with questionable
benefits.
Unless we have some public resources that have tricky robots.txt and are not indexed by
other search engines - this looks like a waste of resources to me.
I might miss something though, so any insight is welcome.

PS: there're 2 messages in the list moderation queue from me that could be deleted -
was fighting with Zimbra identities, sorry for spam.

Thanks.
--
vk

----- Original Message -----
From: "Paweł Kłos" <pawel(a)pawelklos.eu&gt;
To: "Fedora Infrastructure" <infrastructure(a)lists.fedoraproject.org&gt;
Sent: Thursday, August 27, 2015 9:15:09 PM
Subject: Fedora Search Engine

Hi,

The idea of Fedora Search engine has came back with discussion about 
https://fedorahosted.org/fedora-infrastructure/ticket/1055 ticket on the 
meeting.

Many people was truly interested in participating in that project.

So, if somebody is still interested I propose to make some brainstorm 
about that matter, so we could reach better results than years ago :)

First  we can discuss in that mail thread and move then to irc or some 
other place.

Regards,
Pawel Klos
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Susruthan Seran]

Hi,

I agree with you Vasyl Kaigorodov. Even I had the same feeling when I saw
the mail about the new search engine. We could use the existing search
engines. They are doing a good job.

Regards,
*seran.*

On 29 August 2015 at 18:17, Vasyl Kaigorodov <vasyl(a)redhat.com&gt; wrote:

>  Hi folks,
>
>  I did follow the discussion during the meeting, and was hesitating to ask:
>  why exactly do we want own search implemented?
>  From my experience, Google or DuckDuck do index Fedora resources just
>  fine, and whenever I want to find something - I just use those.
>  Own search engine adds to the cost of support and maintenance, with
>  questionable benefits.
>  Unless we have some public resources that have tricky robots.txt and are
>  not indexed by other search engines - this looks like a waste of resources
>  to me.
>  I might miss something though, so any insight is welcome.
>
>  PS: there're 2 messages in the list moderation queue from me that could be
>  deleted - was fighting with Zimbra identities, sorry for spam.
>
>  Thanks.
>  --
>  vk
>
>  ----- Original Message -----
>  From: "Paweł Kłos" <pawel(a)pawelklos.eu&gt;
>  To: "Fedora Infrastructure" <infrastructure(a)lists.fedoraproject.org&gt;
>  Sent: Thursday, August 27, 2015 9:15:09 PM
>  Subject: Fedora Search Engine
>
>  Hi,
>
>  The idea of Fedora Search engine has came back with discussion about
>  https://fedorahosted.org/fedora-infrastructure/ticket/1055 ticket on the
>  meeting.
>
>  Many people was truly interested in participating in that project.
>
>  So, if somebody is still interested I propose to make some brainstorm
>  about that matter, so we could reach better results than years ago :)
>
>  First  we can discuss in that mail thread and move then to irc or some
>  other place.
>
>  Regards,
>  Pawel Klos
>  _______________________________________________
>  infrastructure mailing list
>  infrastructure(a)lists.fedoraproject.org
>
>  http://lists.fedoraproject.org/postorius/[e-mail]...
>  _______________________________________________
>  infrastructure mailing list
>  infrastructure(a)lists.fedoraproject.org
>
>  http://lists.fedoraproject.org/postorius/[e-mail]...
>  
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Emiliano Dalla Verde Marcozzi]

>  On 29 August 2015 at 18:17, Vasyl Kaigorodov <vasyl(a)redhat.com&gt; wrote:
>
> > Hi folks,
> >
> > I did follow the discussion during the meeting, and was hesitating to
> > ask: why exactly do we want own search implemented?
> > From my experience, Google or DuckDuck do index Fedora resources just
> > fine, and whenever I want to find something - I just use those.
> > Own search engine adds to the cost of support and maintenance, with
> > questionable benefits.
> > Unless we have some public resources that have tricky robots.txt and are
> > not indexed by other search engines - this looks like a waste of resources
> > to me.
> > I might miss something though, so any insight is welcome.
> >
> > PS: there're 2 messages in the list moderation queue from me that could
> > be deleted - was fighting with Zimbra identities, sorry for spam.
> >
> > Thanks.
> > --
> > vk
> >
> > Mostly agree with Vasyl Kaigorodov. I think it would be great to use 
Custom
Search Engine from Google. It lets you integrate Google
search engine into your site and tweak it a bit, more info at:
https://cse.google.com/cse/all
The integration is super easy, just some lines of javascript.
Best,
Emiliano.

-- 
Broken code @ https://github.com/edvm

<edvm(a)python.org.ar&gt;

_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Kevin Fenzi]

On Sat, 29 Aug 2015 11:08:22 -0300
Emiliano Dalla Verde Marcozzi <edvm(a)fedoraproject.org&gt; wrote:

>  Custom Search Engine from Google. It lets you integrate Google
>  search engine into your site and tweak it a bit, more info at:
>  https://cse.google.com/cse/all
>  The integration is super easy, just some lines of javascript.
>  Best,
>  Emiliano. 
Well, thats fine, except it's not open source and it's not free 
(in either sense). So, its a no go. 

kevin

_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Christopher Meng]

On Thursday, September 3, 2015, Kevin Fenzi <kevin(a)scrye.com&gt; wrote:
>  Well, thats fine, except it's not open source and it's not free
>  (in either sense). So, its a no go.
>  
Yeah but Firefox uses it still.


-- 

Yours sincerely,
Christopher Meng

http://cicku.me

_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Kevin Fenzi]

On Thu, 3 Sep 2015 06:45:20 +0800
Christopher Meng <i(a)cicku.me&gt; wrote:

>  On Thursday, September 3, 2015, Kevin Fenzi <kevin(a)scrye.com&gt;
> wrote:
>  >
>  > Well, thats fine, except it's not open source and it's not free
>  > (in either sense). So, its a no go.
>  >
>  
>  Yeah but Firefox uses it still. 
Sure. Not sure what that has to do with Fedora Infrastructure tho. 

It's their project, they are welcome to do whatever they like. 

(also, newer firefox versions default to yahoo search)

kevin

_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

Fedora Search Engine - open source possibilities

[Zach Villers]

Here are a few possibilities I've found so far;

Apache Solr - http://lucene.apache.org/solr/

       Features - "Solr is a standalone enterprise search server with a 
REST-like API. You put documents in it (called "indexing") via JSON, 
XML, CSV or binary over HTTP. You query it via HTTP GET   and receive 
JSON, XML, CSV or binary results."

       It's a java servlet, which may be a mark against it. There are 
docker images for it. To try out the Velocity Search engine, move the 
Velocity .jar files into the server library, use the techproducts 
solrconfig.xml, and start the server with bin/solr start -e 
techproducts. You can copy in some html files to the exampledocs 
directory using wget to mirror a site, then index the files with 
bin/post command.


Apache Spark - http://spark.apache.org/

     "Apache Spark™ is a fast and general engine for large-scale data 
processing."

      Here is a use case for someone using it as a search engine; 
http://www.maana.io/company/press-release/the-first-and-only-big-data-sea...

      Just in scanning the docs, you can run it with java or python.



OpenSearchServer - http://www.opensearchserver.com/

      Also runs on java - I haven't tried it out yet.



Sphinx - Free open-source SQL full-text search engine -


      http://sphinxsearch.com/docs/current.html
      (Haven't tried this either)



Here is a link to a recent tech blog with few other possibilities; 
http://www.mytechlogy.com/IT-blogs/8685/tech-blogs-top-5-open-source-sear...



-- 
irc #aikidouke
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

Fedora Search Engine - open source possibilities

[Kushal Khandelwal]

Hi,

How about using elasticsearch ( https://www.elastic.co/ ).
It is built on top of apache lucene with a powerful json based query system.



On Fri, Sep 4, 2015 at 11:31 PM, Zach Villers <greyman(a)zanshin.xyz&gt; wrote:

>  Here are a few possibilities I've found so far;
>
>  Apache Solr - http://lucene.apache.org/solr/
>
>        Features - "Solr is a standalone enterprise search server with a
>  REST-like API. You put documents in it (called "indexing") via JSON, XML,
>  CSV or binary over HTTP. You query it via HTTP GET   and receive JSON, XML,
>  CSV or binary results."
>
>        It's a java servlet, which may be a mark against it. There are
>  docker images for it. To try out the Velocity Search engine, move the
>  Velocity .jar files into the server library, use the techproducts
>  solrconfig.xml, and start the server with bin/solr start -e techproducts.
>  You can copy in some html files to the exampledocs directory using wget to
>  mirror a site, then index the files with bin/post command.
>
>
>  Apache Spark - http://spark.apache.org/
>
>      "Apache Spark™ is a fast and general engine for large-scale data
>  processing."
>
>       Here is a use case for someone using it as a search engine;
>
> http://www.maana.io/company/press-release/the-first-and-only-big-data-sea...
>
>       Just in scanning the docs, you can run it with java or python.
>
>
>
>  OpenSearchServer - http://www.opensearchserver.com/
>
>       Also runs on java - I haven't tried it out yet.
>
>
>
>  Sphinx - Free open-source SQL full-text search engine -
>
>
>       http://sphinxsearch.com/docs/current.html
>       (Haven't tried this either)
>
>
>
>  Here is a link to a recent tech blog with few other possibilities;
>
> http://www.mytechlogy.com/IT-blogs/8685/tech-blogs-top-5-open-source-sear...
>
>
>
>  --
>  irc #aikidouke
>  _______________________________________________
>  infrastructure mailing list
>  infrastructure(a)lists.fedoraproject.org
>
>  http://lists.fedoraproject.org/postorius/[e-mail]...
>  
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Kevin Fenzi]

On Sat, 29 Aug 2015 18:31:29 +0530
Susruthan Seran <susruthanz(a)gmail.com&gt; wrote:

>  Hi,
>  
>  I agree with you Vasyl Kaigorodov. Even I had the same feeling when I
>  saw the mail about the new search engine. We could use the existing
>  search engines. They are doing a good job. 
Well, we could. However that means we depend on a 3rd party non
free/open source service. Also, if we want to adjust results (say to
favor current releases) we may not have control to do that. 

Of course we have gotten by for all these years without a local search
engine, so perhaps we just continue to do so. 

The parts of the project that really expressed interest in a local
search engine are docs and wiki. We could see if the new wiki version
we are going to move to soon has a better search. The docs folks also
are looking at moving to a new platform, perhaps it would have better
search too...

kevin

_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Vasyl Kaigorodov]

On Sat, 29 Aug 2015, Kevin Fenzi wrote:

>  free/open source service. Also, if we want to adjust results (say to
>  favor current releases) we may not have control to do that.  
Hm, adjusting results is the feature I did not think of, that indeed
might be useful.

>  The parts of the project that really expressed interest in a local
>  search engine are docs and wiki. We could see if the new wiki version
>  we are going to move to soon has a better search. The docs folks also
>  are looking at moving to a new platform, perhaps it would have better
>  search too... 
So, what is that new wiki version will be, and what platform are docs
folks are moving to? We could probably do some homework and see what
those have in term of search, and then we can decide if we need a
separate engine, or we can somehow tune wiki/docs to have better
search.

-- 
Vasyl Kaigorodov | Red Hat Product Security
PGP:  0xABB6E828 A7E0 87FF 5AB5 48EB 47D0 2868 217B F9FC ABB6 E828
Free/Busy status: https://url.corp.redhat.com/vk-free-busy-status

_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...

[Zach Villers]

In the meeting, someone brought up Apache Solr as a possible solution.
I've just started playing with it some and need to read the docs a bit
more, but there is a built in search that may fit our needs;

https://cwiki.apache.org/confluence/display/solr/Velocity+Search+UI

I have a pretty busy week, but will try to dig into it a little bit
more. However, if anyone else wants to run with it, please don't
hesitate.

-Zach


On Sat, 2015-08-29 at 17:14 -0600, Kevin Fenzi wrote:
>  On Sat, 29 Aug 2015 18:31:29 +0530
>  Susruthan Seran <susruthanz(a)gmail.com&gt; wrote:
>  
>  > Hi,
>  > 
>  > I agree with you Vasyl Kaigorodov. Even I had the same feeling
>  > when I
>  > saw the mail about the new search engine. We could use the existing
>  > search engines. They are doing a good job.
>  
>  Well, we could. However that means we depend on a 3rd party non
>  free/open source service. Also, if we want to adjust results (say to
>  favor current releases) we may not have control to do that. 
>  
>  Of course we have gotten by for all these years without a local
>  search
>  engine, so perhaps we just continue to do so. 
>  
>  The parts of the project that really expressed interest in a local
>  search engine are docs and wiki. We could see if the new wiki version
>  we are going to move to soon has a better search. The docs folks also
>  are looking at moving to a new platform, perhaps it would have better
>  search too...
>  
>  kevin
>  _______________________________________________
>  infrastructure mailing list
>  infrastructure(a)lists.fedoraproject.org
>  http://lists.fedoraproject.org/postorius/[e-mail]
>  roject.org 
_______________________________________________
infrastructure mailing list
infrastructure(a)lists.fedoraproject.org
http://lists.fedoraproject.org/postorius/[e-mail]...