17
$\begingroup$

This question from March 2020 points out that many questions/answers link to papers at the http://front.math.ucdavis.edu/ frontend instead of at the arXiv itself, but that the frontend had been down a few weeks. There are c.600 affected questions and it looks like a large proportion of the links could be fixed by a simple search-and-replace; so comments and the well-received answer suggested asking SE employees to do such a search-and-replace fix (moderators don’t have that ability), but felt “down for a few weeks” was too soon.

The frontend has now been down for 17 months; presumably this is no longer too soon. Can moderators now petition SE for a search-and-replace fix? Or if that’s too administratively difficult, should we just fix the links by hand, trying not to bump too many at once as usual?

Useful search queries, copied from the older question:

Search-and-replace suggestion: The links fall into several groups, some that should be easy to recognise and fix by regex, others less clear. Hopefully a large proportion can be fixed by regex, leaving the rest to fix by hand.

For links to preprints after April 2007, the correspondence is straightforward — we want to replace a url like http://front.math.ucdavis.edu/0809.1179 with https://arxiv.org/abs/0809.1179, and the first four digits represent YYMM. So a regex replacing https?://front.math.ucdavis.edu/([12]|0[8-9]|071|070[4-9]) with https://arxiv.org/abs/\1 should recognise and fix these links, I think. (Details depend on the regex flavour used, of course.)

Links to older preprints are less consistent. Some have the form https://front.math.ucdavis.edu/math.LO/9909115, and these should be easy again: the identifier part agrees with the arXiv’s, so a regex replace of https?://front.math.ucdavis.edu/(math[.a-zA-Z]*/\d+) by https://arxiv.org/abs/\1 should recognised and fix these. But others use a different identifier, whose correspondence with the arXiv’s identifiers seems unclear — e.g. http://front.math.ucdavis.edu/0310.5109 corresponds to https://arxiv.org/abs/math/0310109 (and https://arxiv.org/abs/0310.5109 doesn’t work). Does anyone else know or see how the correspondence works for these?

Finally there are links to category searches, and the like, which I don’t think we can hope to fix by regex. This edit history has examples of all three kinds.

$\endgroup$
15
  • $\begingroup$ You should add a suggested solution, e.g. a regex describing a good replacement strategy. $\endgroup$
    – Asaf Karagila Mod
    Aug 25 at 9:16
  • 2
    $\begingroup$ I will add that there are 248 comments with such links. (If Stack Exchange would be willing to use some automated proces to change such links, they can probably do so also in comments, not only in posts.) $\endgroup$ Aug 25 at 9:22
  • 1
    $\begingroup$ @AsafKaragila: I’ve added a regex suggestion which I think should get a large proportion of the broken links, and hopefully no false positives, although of course I haven’t been able to test/debug it on the full input set. $\endgroup$ Aug 25 at 9:43
  • 2
    $\begingroup$ Thanks! With the status-review, I hope we'll attract the attention of some StackOverlords soon. I'll see if I can speed it up as well. $\endgroup$
    – Asaf Karagila Mod
    Aug 25 at 9:51
  • $\begingroup$ @PeterLeFanuLumsdaine Are you sure that this revision history is an example you want? This seems to be link to a search rather than a link to and article in the "old format". I left a few more comment in MO editors' lounge. $\endgroup$ Aug 25 at 10:07
  • $\begingroup$ @MartinSleziak: The edit diff in that link shows three different kinds of example: one search link, which I don’t think we could hope to fix by regex; two of the post-April-2007 links that my suggested regex should fix; and two of the pre-April-2007 links that might or might not be regex-able. $\endgroup$ Aug 25 at 10:32
  • $\begingroup$ My mistake. I thought that by "pre-2007" you mean links of the form similar to https://front.math.ucdavis.edu/math.LO/9909115. $\endgroup$ Aug 25 at 10:36
  • $\begingroup$ Looking at only the posts (not comments) I count about 900 links. Of them, more than 250 start with 00-06. And about 60 start with 92-99. So I guess that there are at least 300 links where it's unclear how to transform them. @AsafKaragila Wouldn't it be better to contact SE staff only after we know how the URLs can be transformed? Or is the intention to find out whether this is possible in principle and to deal with the problematic parts later? $\endgroup$ Aug 25 at 11:22
  • $\begingroup$ @MartinSleziak: Right — the confusing thing is that for pre-2007 papers, the frontend seems to provide url’s in at least two different forms, some like the example you give, some like the ones in the edit history I linked. For the ones in the edit history, the correspondence with arXiv’s url/identifier is totally unclear to me. For examples like your form, the conversion is clearer — the identifier math.LO agrees with the arXiv, so we just need to change the url prefix — but it would take a little thought to write a regex that’ll recognise url’s of that form without false positives. $\endgroup$ Aug 25 at 11:23
  • 2
    $\begingroup$ @Martin: The [status-review] puts the thing on their todo list, which is probably long enough that it will take a while. We absolutely should figure out the algorithm for replacing until then, though. $\endgroup$
    – Asaf Karagila Mod
    Aug 25 at 11:30
  • 2
    $\begingroup$ Maybe for some of them it is removing .5 and adding /math and replacing .47 by 0 and adding /math-ph? (But I only tested it on a few URLs.) $\endgroup$ Aug 25 at 12:20
  • 1
    $\begingroup$ Shouldn’t the regex for post-2007 include numbers starting with 2 as well? $\endgroup$ Aug 25 at 12:22
  • 2
    $\begingroup$ @EmilJeřábek: Thanks, fixed! @ Martin: yes, from a few examples it looks like the system works something like that. But at least for my part, I wouldn’t feel confident proposing a regex for a half-guessed system without being able to test the regex out a bit. If you feel confident enough to add a regex to handle these then that’d be great! $\endgroup$ Aug 25 at 13:59
  • $\begingroup$ Does somebody know whether lanl.arxiv.org is down only temporarily or whether all links to that domain should be considered dead, too? (This is not directly related to the front.math.ucdavis.edu, but it seems like a similar problem, so I thought that it might be worth mentioning.) $\endgroup$ Sep 11 at 8:03
  • $\begingroup$ Just to note, Martin and I found yet another oddity. The arXiv front identifier "9612.5114" which I thought should be "math/9612114", really points to "dg-ga/9612014". So maybe ***.51** = dg-ga? $\endgroup$ yesterday
4
$\begingroup$

I've reviewed your request and - while I'd generally want to say "yes, we can do this" - in this case, I'm sorry to say that I don't think it will be possible - at least, not without some effort from you all first. I really do think it's important that the links on this site are functional, so I appreciate your bringing this up for discussion.

The tool we have to make this change is a pretty simple find/replace tool that only works for post content, not comments. It only works for URLs and it can't accept RegEx. So, I think you may already see the problem. While I really appreciate the effort that y'all went to to find the various treatments of the URLs, we can't even only update the ones that are an easy swap without changing all of the others since the content needing to be replaced is the same in all cases.

As you might also see, this doesn't mean that we can't ever do this - it means that we'd need you to help us out before we can. Based on the comments, it sounds like about 2/3 of the ~900 links are the "easy" first and second type. If that's the case, you could manually repair the remaining ~300 URLs first and we can use the link editing tool to fix the remainder of the links which, despite being in two formats require the same change to happen since arXiv helpfully automatically redirects to the correct URL.

So to put it in other words:

  1. First you all work together to fix the outstanding links of the second and third type manually.

    • Based on a comment from Martin Sleziak it looks like there's about 300 links in the third category. It seems like correcting these links in an automated way may be difficult regardless since it's unclear how they should change.
  2. Once the difficult URLs are fixed, we will bulk find/replace the remaining using the following method:

    • Find http://front.math.ucdavis.edu/ and replace with https://arxiv.org/abs/

As far as I understand, we can not bulk edit all 900 URLs first as it will make the pre-2007 URLs even more "wrong" and may make them more difficult to find and repair at a later point in time.

If you can find a consistent way to convert from one of the pre-2007 URLs to the new arXiv URL, you may be able to have someone create a userscript to update these over time, a few questions per day but that's a big "if".

Please let me know if this is clear or if you have any other questions about this response.

$\endgroup$
5
  • $\begingroup$ As someone whose edit history will show his endless appetite for tidying, I would be happy to be part of such a manual effort, which presumably it would be unobjectionable to spread over time. However, is there any way to do it without meaninglessly promoting hundreds of posts to the front page? $\endgroup$
    – LSpice
    Sep 14 at 3:28
  • $\begingroup$ @LSpice Something like that was discussed several times, you can see various feature requests around this. (I still hope that this is going to be implemented at some point - but AFAICT, at the moment there is no way to edit posts without bumping.) See also: Minor edits, subject to review (on this meta) or Allow non-bumping minor edits, but review them on /review (on Meta Stack Exchange). $\endgroup$ Sep 14 at 5:10
  • $\begingroup$ The tool we have to make this change is a pretty simple find/replace tool that only works for post content, not comments. It only works for URLs and it can't accept RegEx. You are a software company, so making new tools to interact with your main product should be a normal thing for you. And a regex find/replace engine is something that could have many other applications in additions to this task. I imagine it there are prioritization and cost/benefit issues here; "we don't have the tools to do that" surely is not the whole story. $\endgroup$ Sep 16 at 10:18
  • $\begingroup$ @FedericoPoloni considering the vast list of things that need to be addressed, building a better tool for this specific issue is extremely low priority. $\endgroup$
    – Catija StaffMod
    Sep 16 at 12:18
  • $\begingroup$ @LSpice I've been attacking them slowly, using the search tools in Martin's answer. It would be nice to divide the effort. Perhaps attacking the "Maybe manually, maybe mass converted" ones would be a good place to start, they could be knocked off over a week or so. The pre-2000 ones likewise. The ~200 posts with links from 2000–06 are going to be a slow slog. There may be some overlap, though, which will help—I've seen posts with multiple links from all over the shop. $\endgroup$ yesterday
2
$\begingroup$

At this moment, it is unclear whether a project like this should be done. (After all, until we heard from people running the arXiv frontend, we still should count as a possibility that it will be running again at some point.)

Still, it might be useful to analyze a bit what would the scope of the project be - how many links would need manual editing, how many links can be mass-converted, etc.

I have started this CW answer - please do edit it if you have additional information that might be useful here.

Resources

Links not going to articles

Links which have to be edited manually

  • Ids starting with 9 are before 2000. At the moment, we get 46 posts from search, 46 posts and 12 comments from SEDE.
  • All links starting with front.math.ucdavis.edu/math/ followed by 00 to 06 were before 2007. SEDE returns 194 posts and 63 comments.
  • Ids starting with 0700 to 0703 are also before April 2007. SEDE returns 3 comments (DR: I have replied to these comments with updated links, paper titles and doi where available) and 8 posts.
  • The above includes also the link http://front.math.ucdavis.edu/0208.5107v2 from this answer which doesn't exactly fit the pattern. This seems to be the only instance of linking to the version number.
  • The few instances where front.math.ucdavis.edu is used as a plain text (i.e., it is following neither http:// nor https://) should be treated on case-by-case basis. SEDE: posts. (DR: there is one remaining text instance that is not intended as a link, but as a mention of the arxiv front.) There are also 9 such comments. (DR: I have gone through and replied to the offending comments with updated links in all cases, and added paper titles and also doi links to the published versions where available)
  • There is only one post with front.math.ucdavis.edu/abs/ (search, SEDE) - a single post can be dealt with manually.
  • There are only a few posts with front.math.ucdavis.edu/q-alg/ - they can be dealt with manually. (A few have already been edited.)

Maybe manually, maybe mass converted

Links with math.AG, math.AT, math.LO etc.: They probably can be converted automatically, but a separate replacement for each category would be needed. Considering the low count, maybe manual editing might be preferable. SEDE returns 31 posts (top 2 now fixed) and 6 comments

Links which can be mass-converted

$\endgroup$
2
  • $\begingroup$ I have struck of a couple that needed only two post edits. We should be able to get some of the items here, with only a few posts each, cleaned up within a couple of days, then likewise clean up the list. $\endgroup$ Sep 20 at 5:30
  • $\begingroup$ If anyone else wants to help, the ones in the "Maybe manually, maybe mass converted" section could be responded to with comments pointing out the updated link, even if the links aren't edited in. $\endgroup$ 14 hours ago

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .