Updated: Is it time to replace links to the UCDavis arXiv frontend?

Question

This question from March 2020 points out that many questions/answers link to papers at the http://front.math.ucdavis.edu/ frontend instead of at the arXiv itself, but that the frontend had been down a few weeks. There are c.600 affected questions and it looks like a large proportion of the links could be fixed by a simple search-and-replace; so comments and the well-received answer suggested asking SE employees to do such a search-and-replace fix (moderators don’t have that ability), but felt “down for a few weeks” was too soon.

The frontend has now been down for 17 months; presumably this is no longer too soon. Can moderators now petition SE for a search-and-replace fix? Or if that’s too administratively difficult, should we just fix the links by hand, trying not to bump too many at once as usual?

Useful search queries, copied from the older question:

• posts containing url:"*front.math.ucdavis.edu*"
• posts containing it that have recently been bumped.

Search-and-replace suggestion: The links fall into several groups, some that should be easy to recognise and fix by regex, others less clear. Hopefully a large proportion can be fixed by regex, leaving the rest to fix by hand.

For links to preprints after April 2007, the correspondence is straightforward — we want to replace a url like http://front.math.ucdavis.edu/0809.1179 with https://arxiv.org/abs/0809.1179, and the first four digits represent YYMM. So a regex replacing https?://front.math.ucdavis.edu/([12]|0[8-9]|071|070[4-9]) with https://arxiv.org/abs/\1 should recognise and fix these links, I think. (Details depend on the regex flavour used, of course.)

Links to older preprints are less consistent. Some have the form https://front.math.ucdavis.edu/math.LO/9909115, and these should be easy again: the identifier part agrees with the arXiv’s, so a regex replace of https?://front.math.ucdavis.edu/(math[.a-zA-Z]*/\d+) by https://arxiv.org/abs/\1 should recognised and fix these. But others use a different identifier, whose correspondence with the arXiv’s identifiers seems unclear — e.g. http://front.math.ucdavis.edu/0310.5109 corresponds to https://arxiv.org/abs/math/0310109 (and https://arxiv.org/abs/0310.5109 doesn’t work). Does anyone else know or see how the correspondence works for these?

Finally there are links to category searches, and the like, which I don’t think we can hope to fix by regex. This edit history has examples of all three kinds.

Answer 1

Thanks for raising this one to our attention. We've now executed the requested changes on Math Overflow. However, it looks like, given the work y'all put in, this change set is relatively issue-free, which is pretty fantastic. Still, as with all link replacements, we do expect a bit of routine cleanup work, so be on the lookout for any cases that look odd.

Let us know if you run into any major fallout as a result of the change and we'll see what we can do to help.

Answer 2

As of 29 March 2022, I claim all the links in questions and answers to front.math.ucdavis.edu that predate April 2007, when the arXiv moved to the new, uniform identifiers YYMM.NNNN[N], have been fixed, as well as every link to things with ids up to the end of 2007. This can be verified by checking the struck-through SEDE links to the various older categories in Martin's answer. This might also be verified by using fancier tools, like finding all mentions of the url string, but removing anything that's a comment, and anything where the arXiv id starts with 08, 09 or 1. There should only be the ≤14 posts with ids starting with 07, all of which are fixed, if any.

There are comments with links that point to the older link syntax used by the UCDavis Front, and (hopefully!) all of them are replied to pointing out a working link.

The remaining changes that need to be made are as follows, copied from Martin's answer:

---

Links which can be mass-converted

• Links where the id starts with 08 and 09 are after 2007, so they should be safe. SEDE returns 113 posts and 34 comments. (SEDE - last activity, SEDE - already edited)
• Links where the id starts with 1 are after 2010, I got 208 posts from search, 209 posts and 112 comments from SEDE. (SEDE - last activity, SEDE - already edited)

---

Answer 3

I've reviewed your request and - while I'd generally want to say "yes, we can do this" - in this case, I'm sorry to say that I don't think it will be possible - at least, not without some effort from you all first. I really do think it's important that the links on this site are functional, so I appreciate your bringing this up for discussion.

The tool we have to make this change is a pretty simple find/replace tool that only works for post content, not comments. It only works for URLs and it can't accept RegEx. So, I think you may already see the problem. While I really appreciate the effort that y'all went to to find the various treatments of the URLs, we can't even only update the ones that are an easy swap without changing all of the others since the content needing to be replaced is the same in all cases.

As you might also see, this doesn't mean that we can't ever do this - it means that we'd need you to help us out before we can. Based on the comments, it sounds like about 2/3 of the ~900 links are the "easy" first and second type. If that's the case, you could manually repair the remaining ~300 URLs first and we can use the link editing tool to fix the remainder of the links which, despite being in two formats require the same change to happen since arXiv helpfully automatically redirects to the correct URL.

So to put it in other words:

1. First you all work together to fix the outstanding links of the second and third type manually.

   • Based on a comment from Martin Sleziak it looks like there's about 300 links in the third category. It seems like correcting these links in an automated way may be difficult regardless since it's unclear how they should change.

2. Once the difficult URLs are fixed, we will bulk find/replace the remaining using the following method:

   • Find http://front.math.ucdavis.edu/ and replace with https://arxiv.org/abs/

As far as I understand, we can not bulk edit all 900 URLs first as it will make the pre-2007 URLs even more "wrong" and may make them more difficult to find and repair at a later point in time.

If you can find a consistent way to convert from one of the pre-2007 URLs to the new arXiv URL, you may be able to have someone create a userscript to update these over time, a few questions per day but that's a big "if".

Please let me know if this is clear or if you have any other questions about this response.

Answer 4

As some of you may have noticed, I have a script which repairs broken links / images, with some more options than the mass-replacement tool available to Stack Exchange staff. It can do all kinds of substitutions, checks if the new URL actually exists and if not, queries the Wayback Machine for a snapshot.

I fed it the following rules:

    {
        "urlPart": "front\\.math\\.ucdavis\\.edu/(0[0-6]\\d{2})\\.5(\\d{3})",
        "status": "KNOWN_BROKEN",
        "replacements": [
            "arxiv.org/abs/math/$3$4"
        ]
    },
    {
        "urlPart": "front\\.math\\.ucdavis\\.edu/(0[0-6]\\d{2})\\.47(\\d{2})",
        "status": "KNOWN_BROKEN",
        "replacements": [
            "arxiv.org/abs/math-ph/$30$4"
        ]
    },
    {
        "urlPart": "front\\.math\\.ucdavis\\.edu/(\\d{4}\\.\\d{4})",
        "status": "KNOWN_BROKEN",
        "replacements": [
            "arxiv.org/abs/$3"
        ]
    },
    {
        "urlPart": "front\\.math\\.ucdavis\\.edu/math(\\.LO)?/(\\d{7})",
        "status": "KNOWN_BROKEN",
        "replacements": [
            "arxiv.org/abs/math/$4"
        ]
    }

I just did a dry run and these are the replacements it would make. The first column is the number of posts the link appears in. I can't post the results here because the table is too large (an answer is limited to 30,000 characters).

The only link it cannot replace is http://front.math.ucdavis.edu/0309.3208 which appears in number-theoretic congruences with geometry and topology?.

When everything checks out, I can limit the edits it makes to posts which cannot be repaired by the mass-replacement tool; after all, my edits will bump posts to the homepage.

Answer 5

At this moment, it is unclear whether a project like this should be done. (After all, until we heard from people running the arXiv frontend, we still should count as a possibility that it will be running again at some point.)

Still, it might be useful to analyze a bit what would the scope of the project be - how many links would need manual editing, how many links can be mass-converted, etc.

I have started this CW answer - please do edit it if you have additional information that might be useful here.

Resources

• Sometimes even a simple search on the main site might give some information.
• SEDE queries for searching for a specific string (or regex) in posts and comments. (If we use a bit more complex query, we can get the same list with recently active questions on the top. In general, editing answer to a questions which has already been bumped for some other reason is better than bumping an old post.)
• List of links from posts and from comments
• Maybe it might be useful to look also at such posts that have already been edited.

Notice that the numbers listed below are about the status at the time when this answer was first posted - you can find how the counts have changed using the links given below and from the updates added to this post.

Links which can be mass-converted

• Links where the id starts with 08 and 09 are after 2007, so they should be safe. SEDE returns 113 posts and 34 comments. (SEDE - last activity, SEDE - already edited)
• Links where the id starts with 1 are after 2010, I got 208 posts from search, 209 posts and 112 comments from SEDE. (SEDE - last activity, SEDE - already edited)

---

_{(DR: I claim all the rest are dealt with.)}

Links which can be mass-converted

• Ids starting with 0704 to 0709 can most likely be converted automatically. At the moment, we get 20 posts and 8 comments (all replied to with a working link). (SEDE - last activity, SEDE - already edited)

• All ids starting with 071 are also after April 2007. Search returns 4 posts (DR: I also gave doi links where possible). SEDE gives 4 posts and one comment. (DR: this has a reply pointing out the updated link) (SEDE - last activity, SEDE - already edited)

Links which have to be edited manually

• All links starting with front.math.ucdavis.edu/math/ followed by 00 to 06 were before 2007. SEDE returns 195 posts. SEDE - last activity, SEDE - already edited

• All links starting with front.math.ucdavis.edu/math/ followed by 00 to 06 were before 2007. SEDE returns 63 comments (all replied to with a working link). SEDE - last activity, SEDE - already edited

• Ids starting with 9 are before 2000. At the moment, we get 41 posts from search, and from SEDE 43 posts and 12 comments (DR: all of these comments have a reply pointing out the updated link and title of the paper) SEDE - last activity, SEDE - already edited

• Ids starting with 0700 to 0703 are also before April 2007. SEDE returns 3 comments (DR: I have replied to these comments with updated links, paper titles and doi where available) and 8 posts. SEDE - already edited

• The above includes also the link http://front.math.ucdavis.edu/0208.5107v2 from this answer which doesn't exactly fit the pattern. This seems to be the only instance of linking to the version number.

• The few instances where front.math.ucdavis.edu is used as a plain text (i.e., it is following neither http:// nor https://) should be treated on case-by-case basis. SEDE: posts. (DR: there is one remaining text instance that is not intended as a link, but as a mention of the arxiv front.) There are also 9 such comments. (DR: I have gone through and replied to the offending comments with updated links in all cases, and added paper titles and also doi links to the published versions where available) SEDE - already edited

• There is only one post with front.math.ucdavis.edu/abs/ (search, SEDE) - a single post can be dealt with manually.

• There are only a few posts with front.math.ucdavis.edu/q-alg/ - they can be dealt with manually. (A few have already been edited.)

Maybe manually, maybe mass converted

• Links with math.AG, math.AT, math.LO etc.: They probably can be converted automatically, but a separate replacement for each category would be needed. Considering the low count, maybe manual editing might be preferable. SEDE returns 27 posts (DR: these have all been manually fixed, I took the opportunity to add more value by giving paper details and links to published versions and also fix other nearby broken links) and 6 comments (DR: all these comments have a reply giving the updated link and title of the paper.) SEDE - last activity, SEDE - already edited
• 6 posts with links to front.math.ucdavis.edu/math/: search, SEDE (SEDE - last activity, SEDE - already edited). There are also 3 such comments. (These comments are listed also here in chat.) Those comments have been edited by Asaf Karagila.

Links not going to articles

• Links to searches (again only a few, DR: only 9 left, all in comments. The intent of the links is clear from the surrounding text or the search query itself, when it has not been responded to with updated info already - I think it is safe to ignore these for now): SEDE - comments, search, SEDE - posts. SEDE - already edited
• There might be some posts which link to the arXiv frontend homepage, such as this answer: Most helpful math resources on the web. That is the only one I get by looking for posts containing front.math.ucdavis.edu/"
• Links to authors (there are only a few of them): search, SEDE - posts. It seems that there are no such links in comments. SEDE - already edited
• Links to front.math.ucdavis.edu/categories - both search and SEDE return one post. And there are two comments. (DR: these comments can be ignored, I think, they both appear in discussion with enough signposting where a reader would be able to figure out how to find the new page) SEDE - already edited
• Link to http://front.math.ucdavis.edu/math in the post: Handling arXiv feeds to avoid duplicates.