Search with CJK keyword with chars ≥ 3 failed with oxygen feedback cloud deployed

[galanohan]

Hi,

To solve the search limitation on CJK keywords with more than 3 characters, as Alin suggested, I've deployed the trial version of oxygen feedback cloud for the webhelp published here: https://cdn.dolphindb.cn/help/index.html

In general, the feedback system looks great. I'm totally satisfied with its backend design, and frontend layout. It's easy to deploy and manage, and also quite handy for readers to put comments to page or block they have questions with.

However, the root cause of deploying a trial version of oxygen feedback system was to solve the search limitation on CJK characters in Webhelp. I've been struggling in oxygen xml editor and config files for weeks, also it might be a solution to put keyword/inderterm tags to every topic, but thinking about the total amount of files exceeds 2000, I believe that's a painful journey to us all ... Then I deployed oxygen feedback cloud.

In the console, I enabled the content indexing and had the content indexed for several times:

image.png (24.68 KiB) Viewed 4849 times

But still, in our online help, https://cdn.dolphindb.cn/help/index.html, if you pick any random chapter titles with more than 3 characters and use them as keywords, no results returned.

image.png (92.72 KiB) Viewed 4849 times

image.png (12.71 KiB) Viewed 4849 times

image.png (12.76 KiB) Viewed 4849 times

Give it a try if you can. You don't really need to understand Chinese, just pick any 3-Chinese-character string, paste them to the search bar and hit the search button.
Or maybe some of my configuration in the backend was wrong? Let me know if I need to provide any further details that could help solve the problem.

Thanks and Regards,
Galano

[alin]

Hi Galano,

Thank you for your feedback!
We are currently investigating the cause of this issue. We will update this thread when we come to conclusion.

Regards,
Alin

[galanohan]

Thank you!

[galanohan]

@Alin

In case you guys test the feature over the url I provided above, let me know if I need to turn on the content indexing in the site configuration.
I turned it off hours ago, because I found when it's on, even the English search results cannot be displayed. When it's off, at least searching English keywords works well as expected.

[galanohan]

@Alin

One question from our readers: to add comments on a page, does a guest need to register over oxygen's site? or does it depend on the license of oxygen feedback? I noticed the ultimate version of oxygen feedback has no limit on moderators.

[alin]

Oxygen Feedback requires users to be authenticated so that they can post comments. The subscription does not have any restriction for the number of commenters.

In addition to users roles that have administration and moderation rights, the following roles are available:

• Regular User
  • has rights to post new comments
  • their comments require approval
• Trusted - comments are automatically approved
• Banned - cannot post comments

When a user creates their account from the Oxygen Feedback Commenting Component embedded on a website they are assigned with the Regular User Role. The new account is created in the same organization where the current site configuration was defined.

[galanohan]

Thank you! I'll inform my colleagues who weren't granted admin roles to give it a test.

[alin]

Hello,

We have released a new version of Oxygen Feedback (v. 3.0.3). In this version we have tried to improve the search engine to return better results for Chinese content. Please give it a try!

Regards,
Alin

[galanohan]

It works very well!

[galanohan]

Another problem comes up:

If I set the content indexing language as Chinese, then function names, parameter names or keywords in SQL expression cannot be searched, as they were processed as English. Is there a way to index two languages?

[alin]

Hello,

Please send us a minimal DITA sample on which the problem can be reproduced.

Regards,
Alin

[galanohan]

md_dita_zh_en.zip

(19 KiB) Downloaded 578 times

Hi Alin,

please find the sample package from attached.

image.png (4.79 KiB) Viewed 4574 times

[Costin]

Hi Galano,

Thank you for the feedback you provided and for the sample! We were able to reproduce the behavior you reported using your sample.
We are currently prepairing to release a new version of the Oxygen Feedback, so almost all of our HR are focused on implementing the features to be included in the new release, which should Occur in late October.
However, I have just logged an issue in our internal issues/improvements tracking system, altogether with the feedback and the same files you provided.
After the new version release, our development team will look into the issue and try to find a fix to be implemented in a future version of the Oxygen Feedback.
We will notify this thread when/if a new implementation will become available.

Regards,
Costin

[galanohan]

A workaround I could think of to bypass the issue I reported is:
1. write a script that split any given CJK keywords for search into multiple groups separated by blank space. For example, like this:

image.png (23.34 KiB) Viewed 4504 times

2. When the on-click event occurs after user clicking the search button, the script above is triggered.
3. The search results returned by the script in step 1 is then returned to the search result page.

[beniamin_savu]

Hi,

Indeed, it seems that with this workaround it may be possible to add the spaces required by the search engine to work.

Regarding this aspect, we managed to create a sample publishing template that changes the search query after the user clicks the search button. In this example we added the string "modified" to the end of the search query entered by the user. The zip file for this sample publishing template can be found here:

ChangeSearchQuery.zip

(195.26 KiB) Downloaded 682 times

You can use this publishing template as a starting point to modify the search query according to your needs. Please note that this will require a certain level of JavaScript expertise. To modify the functionality you will have to edit the custom.js file found at: ${pubishingTemplateFolder}/js/custom.js

Best regards,
Beniamin Savu
Oxygen WebHelp Team
http://www.oxygenxml.com

[galanohan]

Hi Benjamin,

Thanks for the custom.js. I modified line 13 to the following:

var executed = false;
$( document ).ready(function() {
$("#searchForm").on("submit", (e) => {
// triggers the submit event handler multiple times.
if(!executed) { // execute it only one time.
e.stopPropagation();
var userQuery = $('#textToSearch').val();
if (userQuery.trim() === '') {
e.preventDefault();
return false;
}

//userQuery = userQuery.replace(/[\u4e00-\u9fa5]{2}/g, '$& '); // split every given Chinese input by separating every two Chinese characters by spaces.
userQuery = userQuery.replace(/[\u4e00-\u9fa5]/g, '$& '); // split every given Chinese input by separating every Chinese character by spaces.
$('#textToSearch').val(userQuery);
executed = true;
}
});
});

As you can see,

//userQuery = userQuery.replace(/[\u4e00-\u9fa5]{2}/g, '$& '); // split every given Chinese input by separating every two Chinese characters by spaces.
userQuery = userQuery.replace(/[\u4e00-\u9fa5]/g, '$& '); // split every given Chinese input by separating every Chinese character by spaces.

Both works pretty good. But given the fact that CJK string with no more than 2 chars as search input has no problem in text search, string split by every 2 chars seems to be able to return more accurate results, especially when the keyword for search contains only 2 chars.

Thanks so much!

Btw, now the local webhelp search is solved by this custom.js, will the webhelp output conflict with the content indexing in oxygen feedback if the content indexing is based on Chinese or English?

[cosmin_andrei]

Hello,

will the webhelp output conflict with the content indexing in oxygen feedback if the content indexing is based on Chinese or English?

Could you please provide more details in order to better understand your use-case?

[galanohan]

For example, with the help the preceding custom.js, search with Chinese string with more than 2 characters is now available in local Webhelp output.
Now, if I deploy the Chinese webhelp package to our company site, and then specify the directory of index.html at a certain site in oxygen feedback, enable the content indexing, and set the indexing lanuage as Chinese, and save the configuration. When the indexing completes for the content in the directory specified in oxygen feedback site, the text search in the specified URL will use the text search provided by the oxygen feedback, and text search supported by js/custom.js in the publishing template will be overridden, did I understand the login right?

[beniamin_savu]

Hi,

The custom.js file is executed regardless of how the WebHelp output is deployed, locally or on a server. Oxygen Feedback's search function will not override the behaviour of the custom.js file when used in the WebHelp output. The custom.js will still be executed and will split the words according to your changes.

We suggest that the custom.js file to be applied only if the WebHelp output is run locally.

Best regards,
Beniamin Savu
Oxygen WebHelp Team
http://www.oxygenxml.com

[galanohan]

Thanks for the advice!