14
$\begingroup$

One can see from the recent discussion Correcting links with university proxy (especially MathSciNet links) that some users add in their posts links that only work for people behind a specific proxy, and that some other users spend time correcting those links.

This looks like a problem that can be solved automatically. I would suggest Stack Exchange to implement a check upon posting a question/answer: if a link coming from a specific blacklist of proxies is detected, a warning is shown ("This post may contain links to documents that are not visible by users outside your institution; do you wish to continue?")

$\endgroup$
3
  • 2
    $\begingroup$ That's a good feature request, hopefully the fact there are several academic sites on the network will encourage the SE Overlords to implement it too. $\endgroup$
    – Asaf Karagila Mod
    Commented Jul 21, 2017 at 8:43
  • $\begingroup$ If it turns out that we like this suggestion here, then it should be posted for discussion on the "big" stackexahnge meta. $\endgroup$
    – Gerald Edgar
    Commented Jul 21, 2017 at 12:26
  • 2
    $\begingroup$ Probably diamond users know more about this, but I think that it is already possible to add warnings triggered by certain strings or to blacklist them entirely. For example, see the list of blacklists and warnings on math.SE. However, if also rewrite of the link should be implemented, that would probably be a new feature. $\endgroup$
    – Martin Sleziak
    Commented Jul 22, 2017 at 4:22

3 Answers 3

Reset to default
5
$\begingroup$

I've added a warning similar to what Martin Sleziak suggests:

Warning for proxy

The warning looks for words1 that include the string proxy and later .edu2. You can see the regex I used along with some tests at regex101.com. It shouldn't catch examples of "proxy" found in the post body. Unfortunately, it doesn't pick up all listed proxy links nor does it rewrite the URL for the user.

Please let me know if there are any problems with this warning or if I should change the regex to warn about a wider range of URLs.

Footnotes:

  1. As defined as a consecutive string of letters, numbers, dashes and/or periods.
  2. For this site, most proxies seem to be associated with educational institutions.
$\endgroup$
8
  • $\begingroup$ Thanks, this looks exactly like what I had in mind! $\endgroup$
    – Federico Poloni
    Commented Feb 19, 2019 at 6:22
  • $\begingroup$ It's very minor, but there seems to be a subject-verb agreement mistake in the error message (your links work... others can follow it). $\endgroup$
    – Federico Poloni
    Commented Feb 19, 2019 at 6:23
  • 1
    $\begingroup$ Oh. Good catch. I played around with that message for a bit and didn't give it the final pass it needed, I suppose. I changed the final word to "them". $\endgroup$
    – Jon Ericson StaffMod
    Commented Feb 19, 2019 at 7:39
  • 1
    $\begingroup$ "For this site, most proxies seem to be associated with educational institutions." and .edu is almost exclusively associated with educational institutions from the United States. $\endgroup$
    – Najib Idrissi
    Commented Feb 19, 2019 at 15:30
  • 1
    $\begingroup$ @NajibIdrissi: True. For the purposes of this warning, however, I need a text pattern that has a high probability of indicating a proxy link and a low probability of creating more confusion. I'm happy to add more checks if they seem to fit that criteria. $\endgroup$
    – Jon Ericson StaffMod
    Commented Feb 19, 2019 at 16:30
  • $\begingroup$ To me, it seems unlikely that any non-proxy URL would include the phrase "proxy". Maybe just search for that? (I say this as someone who has accidentally included a proxy URL in an answer and whose institution does not have ".edu" in its URL. $\endgroup$
    – Hugh Thomas
    Commented Feb 19, 2019 at 20:11
  • 2
    $\begingroup$ @HughThomas: The regex doesn't really know if the string is a URL. Doing that would be a little more complex. That said, I could search for strings that start with https?: and include "proxy". That would knock off the "edu" requirement and avoid false positives. $\endgroup$
    – Jon Ericson StaffMod
    Commented Feb 19, 2019 at 20:24
  • $\begingroup$ @JonEricson: Thanks! I know nothing about this and should probably shut up and let the people like you who understand the issues do their thing. Nonetheless two things occur to me. One is: maybe people neglect to write http; maybe someone with better search skills than me could see how much that happens. Another one is: could the regex look for whatever it is in the markup we use that encodes that something is a clickable link? $\endgroup$
    – Hugh Thomas
    Commented Feb 20, 2019 at 3:44
10
$\begingroup$

It could be reasonable to collect some past examples of such proxies. This would help in assessing whether something like this is technically feasible. (And maybe also to which extent implementing this might actually be useful.)

Also I assume that if something like this is implemented at some point, we'll need to prepare list of regular expressions for such automatic rewrites - so it is good which proxies we are concerned with. (The post is CW, feel free to add more examples.)

Typically the rewrite would look like this:

Some differently looking changes:

$\endgroup$
10
  • $\begingroup$ I will be grateful also for suggestions about the format. Would it be better to omit the examples (or perhaps mention a few of them outside the list) and simply leave there number of occurrences together with a link to SEDE query listing them? $\endgroup$
    – Martin Sleziak
    Commented Dec 28, 2018 at 15:30
  • 7
    $\begingroup$ Martin, on behalf of many MO users, I would like to thank you for your outstanding work on this matter and many others. We all appreciate your efforts very much! $\endgroup$
    – Joel David Hamkins
    Commented Dec 29, 2018 at 23:27
  • $\begingroup$ That adelaide proxy link was due to me, and if you go look at the result, I had edited out the proxy link in a past revision; the data explorer seems to be picking up old versions of questions and answers. $\endgroup$
    – David Roberts Mod
    Commented Dec 30, 2018 at 1:13
  • $\begingroup$ @DavidRoberts If you look at the query, it is on purpose that we are looking on occurrences in all past revisions. After all, the query is called: "Posts with given word in some of the revisions". This should be obvious also from the fact that links listed as examples are to past revisions. (Ideally all such links should be edited by now. Since Federico Poloni suggested some automation, it is good to know how many manual edits would be automated in this way.) $\endgroup$
    – Martin Sleziak
    Commented Dec 30, 2018 at 2:59
  • $\begingroup$ If somebody wants to find the occurrences that still exist, it's possible to use on site search. Of course, you can also use SEDE if you prefer. $\endgroup$
    – Martin Sleziak
    Commented Dec 30, 2018 at 2:59
  • $\begingroup$ And it is also possible to look also for all posts where given string was edited out. $\endgroup$
    – Martin Sleziak
    Commented Dec 30, 2018 at 3:00
  • 1
    $\begingroup$ @Martin ah, I see: to get a scope of the history of the problem, not just the current state. $\endgroup$
    – David Roberts Mod
    Commented Dec 30, 2018 at 8:17
  • 1
    $\begingroup$ @DavidRoberts Yes, that was the intention. Even if the outcome of this will be that the problem is too minor to require some automated solution (at least at current size of MO site), at least the feature request is not left without a response. (As I have already mentioned in the comments under the question, if we want just some kind of warning triggered based on some regex detecting url containing a proxy, that's already possible. However, for this we would need some help from moderators.) $\endgroup$
    – Martin Sleziak
    Commented Dec 30, 2018 at 11:07
  • 1
    $\begingroup$ The UMich proxy has changed so that, instead of something like link.springer.com.proxy.lib.umich.edu , one gets link-springer-com.proxy.lib.umich.edu . (I see that this is consistent with the library.nd.edu and galib.uga.edu proxies.) $\endgroup$
    – LSpice
    Commented Dec 31, 2018 at 17:07
  • 1
    $\begingroup$ @LSpice Unless I missed something, so far this proxy was only used on this site in the "old form". Anyway, I have included the information from your comment into the post. $\endgroup$
    – Martin Sleziak
    Commented Jan 2, 2019 at 20:37
3
$\begingroup$

Let me add another suggestion how to handle this - with already existing tools, without any need to implement something new. (It is probably not worth to create a separate question for this - I am posting this suggestion as an answer, we'll see from the voting and the comments what MO users and moderators think about it.)

It is already possible to show various warnings depending on body, title and tags of the post. As far as I can tell, the warning is triggered depending on presence of some specific strings and it is possible to use regular expressions.1 From what I have seen in meta discussions, such warnings can be added by Stack Exchange staff. The request to do so quite often come from the mods, more often than not after some discussion on meta.

Proposal. I think that it should not be difficult to create a regex to find out whether a post contains an url containing the string proxy. In such case, a warning should be shown (with a short explanation that such links could be a possible problem and suggesting to the poster to check the links).

Disadvantages. This will create some false positives - which is the reason why I do not suggest to blacklist such input, just to show a warning. Also some users will probably ignore the warning - in such cases we have to rely on the fact that somebody notices the post and corrects the link.

Advantages. It is probably easier to create a check which might return some false positives but which should definitely catch most of the problematic links. (I would definitely consider this better than creating a separate regular expression for each proxy - assuming the numbers in the CW answer are close to correct, then by creating a regular expression which in some way handles a specific proxy we would help with at most 5 posts. I don't think that this is worth the effort - MO would need for each specific proxy go through moderators and ask somebody from Stack Exchange to add it to the list.) Unless there is some way which would take care of many proxies at the same time - which seems unlikely to me - this seems to me much simpler solution than trying to implement some way to automatically rewrite such links, considering the number of posts where this is actually a problem.

1If I find some place where this is documented in detail, I can add it here. The best I can provide at the moment is the list on Mathematics Meta. As you can see, it contains also some regular expressions.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .