Indexing, Search on sangham.net (dokuwiki)

[Dhammañāṇa]

Aramika  

Dieses neue Thema (bzw. diese/r Beitrag/e) wurde  aus abgetrennten Beiträgen, ursprünglich in Server space management , hinzugefügt. Für ev. ergänzende Informationen zur sehen Sie bitte das Ursprugsthema ein. Anumodana!

The new topic (or post/s) here are originaly from Server space management . For eventual additionally information: please visit also the Topic of origin. Anumodana!

[Original post:]

Not sure about the impact of log files on search engine, as currently not so success in making use of it.
Giving only results found in title at large.

[Moritz]

Not sure about the impact of log files on search engine, as currently not so success in making use of it.
Giving only results found in title at large.

Vandami Bhante

Not sure I understand correcty. Is it about the sangham.net DokuWiki search function?
Can you give an example of something that is not found but should be found? Maybe then I can find out the reason.

I just tried to search something randomly copy-pasted from some page in the Khmer Tipitaka, and it seems to work: https://sangham.net/index?do=search&id=index&q=បដិច្ចសមុប្បាទវិភង្គោ

[Moritz]

Oh, only in title, I see.
The example page I copied from and pasted to search also only had the title: https://sangham.net/cs-km/tika/abh/mula/abh.vb.06.1_mula_tik

Is this page actually working correctly? Or should there be some text visible in it?
I remember you used the include plugin to include text from other pages, Bhante, but I am not so familiar with all the plugins and how they work. Should there maybe be text included here from another page?
Maybe the search engine would also not find text that is included with the include plugin.

[Moritz]

Hmm, I just tried with some text copied from here https://sangham.net/km/tipitaka/sut/kn/uda/sut.kn.uda.40, which was included as a text from here https://sangham.net/km/tipitaka/book_052.
At first I did not find any results. But after visiting the "original" page https://sangham.net/km/tipitaka/book_052 from where the text was included, and then searching again, the text was found.

So it seems there is missing a lot in the index. Maybe something got lost or invalid by the upgrade.
Normally, whenever opening a page, or editing it in the DokuWiki, it is indexed. If changing files directly on the server, by FTP upload, this is not updated in the index. Maybe same with using BatchEdit, not sure.

There is a plugin that can be used to rebuild the whole DokuWiki index, but as tried out in the past, it takes extremely long to finish for a huge Wiki as this, and one has to leave the browser open all the time.
Not sure if I should clear and rebuild the whole index, or just update it.
I will first take a closer look at how it actually works.

[Dhammañāṇa]

Oh, only in title, I see.
The example page I copied from and pasted to search also only had the title: https://sangham.net/cs-km/tika/abh/mula/abh.vb.06.1_mula_tik

Is this page actually working correctly? Or should there be some text visible in it?
I remember you used the include plugin to include text from other pages, Bhante, but I am not so familiar with all the plugins and how they work. Should there maybe be text included here from another page?
Maybe the search engine would also not find text that is included with the include plugin.

Single release pages have some texts, yes, but the content is fetched from source. Not sure if the include part would be included in search index.

But generally searches bring no more results (correct, seemingly now some) from content but only from first hearder, or pagename, since short. As if index got lost. Or maybe just building anew.

May Nyom try to search word "hut" found in Theragatha for example.

[Dhammañāṇa]

Hmm, I just tried with some text copied from here https://sangham.net/km/tipitaka/sut/kn/uda/sut.kn.uda.40, which was included as a text from here https://sangham.net/km/tipitaka/book_052.
At first I did not find any results. But after visiting the "original" page https://sangham.net/km/tipitaka/book_052 from where the text was included, and then searching again, the text was found.

So it seems there is missing a lot in the index. Maybe something got lost or invalid by the upgrade.
Normally, whenever opening a page, or editing it in the DokuWiki, it is indexed. If changing files directly on the server, by FTP upload, this is not updated in the index. Maybe same with using BatchEdit, not sure.

There is a plugin that can be used to rebuild the whole DokuWiki index, but as tried out in the past, it takes extremely long to finish for a huge Wiki as this, and one has to leave the browser open all the time.
Not sure if I should clear and rebuild the whole index, or just update it.
I will first take a closer look at how it actually works.

If it got lost, than it will be just slowly rebuild unless the page is visited. Seem reasonable.

Nyom remembers his efforts once to make a full? Atma can link the topic here if needing reminder.

(Indexing also include content would be great, but not sure it machine can fetch the complete build page or just takes the txt-content, as the searches display always the wiki-syntax as well and not just text (good and "bad")

[Dhammañāṇa]

That was the challenge topic back 2019 https://forum.sangham.net/index.php?topic=9172.msg17988#msg17988

[Moritz]

Nyom remembers his efforts once to make a full? Atma can link the topic here if needing reminder.

Yes, I remember. Thanks for the link.

Back then it was still on the old server from Greensta with no access to do things on the server directly, other than transfer files via FTP, and trigger PHP scripts through the web browser.
I think it should be possible to index more efficiently with some program running directly on the server, not via this website interface. Probably some way to do that is already built in, just have to find it in the documentation or look at the code.

[Dhammañāṇa]

Maybe a help or anchor: https://www.dokuwiki.org/taskrunner

[Moritz]

Thanks. I will take a look at it.

[Dhammañāṇa]

Aramika  

Ein oder mehrer Beiträge wurden hier im Thema abgeschnitten und damit in neues Thema "Vissakamma, the mansion builder of Devas " eröffnet, dem angehäng.

One or more posts have been cut out of this topic here. A new topic, based on it, has been created as "Vissakamma, the mansion builder of Devas " or attached there.

[Moritz]

The indexer is a bit complicated. So still not done.
But should be able to do it latest till the weekend.

[Moritz]

The indexer is a bit complicated. So still not done.
But should be able to do it latest till the weekend.

Like often, whenever I promse something, it does not work out.
I think it would be good to have the possibility to make a complete index rebuild in the background, but still use the existing index as long as it is not completed. That was the main thing I wanted to do. But so far still not understood enough to make it work. In the meantime I had submitted some tiny improvements to the DokuWiki GitHub, and the maintainer informed me that he is working on a complete rewrite of the indexer, and invited me to look at it and maybe help work on it. So I spent much time on distractions.

Now I am just running the existing indexer update script on the server, which I could have done earlier. I don't understand some error messages in between, but it will probably not break anything.
Should be finished in half a day or so now.

[Dhammañāṇa]

Sadhu for efforts, Nyom.