Image source: The Motley Fool.
The Bank of N.T. Butterfield & Son Limited (NTB 2.48%)
Q2Â 2020 Earnings Call
Jul 23, 2020, 10:00 a.m. ET
Contents:
- Prepared Remarks
- Questions and Answers
- Call Participants
Prepared Remarks:
Operator
Good morning. My name is Isley, and I will be your conference operator today. At this time, I would like to welcome everyone to the Second Quarter 2020 Earnings Call for The Bank of N.T. Butterfield & Son Limited. [Operator Instructions]
I would now like to turn the conference over to Noah Fields, Butterfield's Head of Investor Relations.
Noah Fields -- Head of Investor Relations
Thank you. Good morning, everyone, and thank you for joining us. Today, we will be reviewing Butterfield's second quarter 2020 financial results as well as some information related to COVID-19 and the Bank's operational activities and asset quality.
On the call. I'm joined by Butterfield's Chairman and Chief Executive Officer, Michael Collins; and Chief Financial Officer, Michael Schrum. Following their prepared remarks, we will open the call up for a question-and-answer session. Yesterday afternoon, we issued a press release announcing our second quarter results. The press release, along with the slide presentation that we will refer to during our remarks on this call, are available on the Investor Relations section of our website at www.butterfieldgroup.com.
Before I turn the call over to Michael Collins, I would like to remind everyone that today's discussions will refer to certain non-GAAP measures, which we believe are important in evaluating the company's performance. For a reconciliation of these measures to U.S. GAAP, please refer to the earnings press release and slide presentation. Today's call and associated materials may also contain certain forward-looking statements, which are subject to risks, uncertainties and other factors that may cause actual results to differ materially from those contemplated by these statements. On Slide 25 of the presentation, we have also included a list of potential factors relevant to the implications of COVID-19 for the Bank. Additional information regarding these risks can be found in our SEC filings.
I will now turn the call over to Michael Collins.
Michael Collins -- Chairman and Chief Executive Officer
Thank you, Noah, and thanks to everyone joining the call today. The Bank's second quarter results were strong despite some of the challenges associated with COVID-19. As an essential service provider, the Bank was able to support its clients throughout the health crisis with strict adherence to all recommended health and safety guidelines. We are grateful to our frontline staff who braved uncertainty to help keep our island economies operating, and to our customers for their understanding as we temporarily transitioned to a remote working environment. I've been very impressed with the Bank's operational resilience and our ability to quickly adjust to the new safety guidelines and social distancing rules relevant in these jurisdictions.
As you can see in the summary results on Page 4 of the presentation, we reported net income of $34.3 million or $0.67 per share. In the second quarter, net income was 14.9% lower than the prior quarter, due primarily to the COVID-19-related economic slowdown. Our core return on average tangible common equity was 15.5%, down from 18.6% in the prior quarter. Net interest margin was down 15 basis points to 2.48% compared to the linked quarter and our cost of deposits dropped 28 basis points to 14 basis points.
We believe our capital position of profitability remained strong. And as a result, the board approved a quarterly cash dividend of $0.44 per share, while we continue to repurchase shares throughout the quarter. During the quarter, we successfully issued $100 million of 5.25% 10-year fixed to floating rate subordinated debt, which will mostly be used to replace existing debt. It is beneficial for the Bank to be a known issuer in the debt capital markets, and the latest sub-debt issuance helps us to continue the dialog with fixed income investors.
Turning now to Slide 5. Butterfield operates in 10 locations with each jurisdiction managing through the health crisis relatively well at this point. In Bermuda, the Cayman Islands, Guernsey and Jersey, the suppression of COVID has overall been effective. Bermuda was just entering its tourism season when the pandemic struck, and the island entered a mandatory shelter in place period for the month of April. The island was essentially closed and has gradually reopened in May and June with the airport opening on July 1. The number of flights are significantly below historical norms, but are expected to increase in August and into the fall.
The cruise ship season is effectively lost, which will cost the government passenger fees. And local businesses such as transportation companies, bars, restaurants and retailers will also miss out on significant revenue. With fewer flights to the island, hotels will also be impacted by the limited tourists season, which typically is most active through October.
The timing of shutdowns in Cayman came toward the end of their 2019 tourism season. And they have been in their summer slow season, which has moderated the severity of the impact on their economy so far. They are hoping to open their airport in the next couple of months as they approach the traditional autumn and winter tourist season. The Channel Islands have had more COVID cases due to their proximity to the U.K. and France, but tourism is not as significant a contributor to the economy compared to Bermuda and Cayman where tourism is approximately 17% and 25% respectively.
We've been making significant strides to help support our communities through these challenging times. In addition to local community-based programs, we have offered residential mortgage deferrals to clients in good standing in both Bermuda and Cayman for up to six months and three month interest only options to small and medium-sized businesses. We are also working closely with larger corporate clients who may need to adjust terms of their loans and working capital requirement.
We will continue to look at how we run our business in the near-term and longer term, reviewing costs and potential for new fee-generating opportunities with less reliance on interest rate-sensitive revenue sources. Butterfield's exposure to hotels and restaurants remained limited with well structured loans and collateral packages in place. The remaining mortgage deferrals expire in September after six months, and we will be keeping in close contact with clients and we'll continue to work with customers who may need assistance.
I will now turn the call over to Michael Schrum to provide more details on the second quarter.
Michael Schrum -- Group Chief Financial Officer
Thank you, Michael. Looking now at Slide 7, we provide a summary of net interest income and NIM. In the second quarter, NIM of 2.48% was 15 basis points lower than the prior quarter as global interest rates fell across the yield curve. Lower interest rates pushed down floating rate loans and resulted in loan yields of 4.53%, down 27 basis points. Lower deposit cost did partly offset the lower asset yields.
Looking now at Slide 8. Non-interest income was down 12.4% compared to the prior quarter, due primarily to slower temporary economic activity during government mandated lockdown periods. Banking fees fell as a result of lower card and merchant services fees. Foreign exchange commissions were also lower as activity was naturally subdued during the months of April and May.
On Slide 9, we provide an overview of core non-interest expense, which was down 6.5% in the second quarter compared to the prior quarter. While the first quarter of 2020 had some staff exit cost included in salaries and benefits, the second quarter benefited from lower costs related to consultants, travel, client entertainment and marketing. The core cost to income ratio was 66.7%, primarily due to lower COVID-19-related revenue generation. We continue to target at through cycle efficiency ratio of 60% and expense management will be an important factor in a lower for longer interest rate period.
Slide 10 summarizes regulatory and leverage capital levels. We continue to maintain strong capital levels that are above regulatory requirements. Tangible book value per share increased 3.6% in the second quarter and is up 8.4% in the last two quarters. We view this as an important long-term value creation measure. In addition to the regular quarterly dividend, we have continued to repurchase shares during the quarter. As at the end of the second quarter, we had approximately 950,000 shares remaining in our 3.5 million share repurchase authorization from December of 2019. In addition to supporting the dividend buybacks and organic growth, we also continue to evaluate M&A opportunities.
Turning now to Slide 11. Butterfield's balance sheet has stabilized following the expected deposit declines from the ABN AMRO Channel Islands' acquisition. Deposits at 30 June, 2020 were $11.6 billion, a decrease of 1.2% from the end of the prior quarter. The lower deposit levels resulted primarily from the attrition of euro deposits in the Channel Islands. Overall, the cost of deposits is down 28 basis points to 14 basis points due to term deposit rollovers and further downward alignment of customer deposit rates. The balance sheet continues to be conservative and highly efficient with a low risk density of 37.1% in terms of risk weighted assets to total assets.
On Slide 12, we provide more detail regarding residential mortgages by location, commercial loans by type and timeline of mortgage deferral participation levels. In general, the Bank's loans are manually underwritten by professionals with significant local market experience. 63% of loans are residential and the majority of those are in Bermuda, Cayman and Central London. Approximately 80% of the Bank's residential loans have an LTV of below 70% and our commercial loans have an origination standard of below 65% LTV.
As Michael mentioned earlier, one of the ways we've tried to alleviate the economic impact of COVID-19 on our communities was to offer mortgage deferral plans, which temporarily has put more money in people's pockets to assist recovery efforts. In the months of April through June, 85% of Bermuda and Cayman mortgage holders in good standing benefited from principal and interest payment deferrals with 15% of borrowers electing to continue with principal and interest payments. From July 1, customers have to opt-in if they wanted to continue with another three months of payment deferrals, of which approximately 50% have opted to participate. We have also offered small and medium-sized businesses the option of interest-only payments for the three months starting on April 1. Overall, these assistance programs have been really well received by customers.
As you can see on Slide 13, Butterfield emphasizes low-credit risk in its investment portfolio with 99% of our securities comprised of AAA-rated U.S. government guaranteed agency securities. Non-accrual loans did increase by $20 million in the quarter due to one C&I loan and collateral dispute litigation, and a dozen or so mortgages that were not eligible for deferral at the end of the first quarter. We continue to work with the customers in difficulty during this time. During the second quarter, we increased our CECL estimate by $4.4 million, primarily in the consumer and commercial lending books. The increase was based on lower general economic forecasts in the quarter. And the allowance for credit losses is now at $40.2 million or 79 basis points of total loans.
On Slide 14, we discussed the average cash and securities balance sheet with a summary interest rate sensitivity analysis. Butterfield's duration of weighted average life of the securities portfolio has decreased in the second quarter due to lower U.S. interest rates, which caused higher prepayment speeds in our agency securities book. The sensitivity of the book has increased in both rising and falling interest rate scenarios. In the case of falling rates, we included the assumption that we would charge negative rates on deposits if rates go below zero. The Bank's unrealized gains are currently at $180 million, up from $154 million at the end of the first quarter and $59 million at the end of 2019. We believe this will help moderate compression of the securities book yields in the years to come.
I will now turn the call back to Michael Collins.
Michael Collins -- Chairman and Chief Executive Officer
Thank you, Michael. You may have seen in our recently filed AGM proxy statement that the Bank is proposing to add two new directors. The first is Leslie Godridge, who has over 40 years of financial experience and was most recently at U.S. Bancorp, where she was Vice Chairman, Director U.S. Bank NA and Co-Head, Corporate and Commercial Banking, following a number of years in senior positions at Bank of New York. The board will also be joined by Jana Schreuder, who has recently retired as Chief Operating Officer of Northern Trust Corporation, after a career which began at Northern Trust in 1980. I am delighted to be adding two highly experienced and confident financial professionals to our board, which will now have 11 directors. These two new directors are on the slate for the scheduled AGM on August 12.
Due to travel restrictions and health concerns around large group meetings in this new normal, the board has decided to hold a Virtual Annual General Meeting for 2020. Participation details for shareholders will be sent out shortly. As we manage through this period of change and uncertainty, we will continue to review all aspects of our businesses, products and jurisdictions to determine where we can improve efficiency. This will strengthen the Butterfield franchise to benefit all stakeholders and continue to provide industry-leading returns.
Thank you. And with that, we'd be happy to take your questions. Operator?
Questions and Answers:
Operator
[Operator Instructions] Our first question today comes from Michael Perito with KBW.
Michael Perito -- KBW -- Analyst
Hey, good afternoon, guys.
Michael Collins -- Chairman and Chief Executive Officer
Good morning, Mike.
Michael Perito -- KBW -- Analyst
Thanks for taking the questions. I had a couple of things I want to address. I guess first, just on the expense side, obviously, a pretty nice sequential drop here. I'm curious maybe if you can give us -- Michael, you kind of alluded in the prepared remarks that there is some ongoing monitoring given that the lower rate environment, etc. But just curious maybe if you can give us a little bit more color on what some of the major drivers of the sequential decrease were and the sustainability of those items? And then also, just more broadly, what are some of the things -- I know you guys have been fairly proactive managing the expense base already. So just curious what some of the other areas that you guys are potentially looking at that we should be mindful of moving forward?
Michael Schrum -- Group Chief Financial Officer
Yeah. Good morning, Mike. It's Michael Schrum. So maybe I'll start off just on the sequential a bit and then Michael can sort of talk about the longer term. So I think, I mean, looking at Q2, obviously there is some temporary reductions just because we're obviously not out meeting with clients and attending events, etc. Travel clearly is very subdued at this point in time. And then tactically, we've also slowed down some of the brand rollout expenses to kind of lower the burn rate. You'll recall, we redesigned a brand for Butterfield in 2019. That rollout schedule was originally compressed and we've sort of moved that out a bit to just use more internal resources really for that.
If I look at sort of sequentially, if we wanted to normalize last quarter, we sort of landed the final bit on the staffing levels following the ABN acquisition. So there were some severance costs and staff exit costs included there. So it's probably sort of too heavy last quarter and it's probably, if I think about this quarter and trying to normalize that, it's probably about $2 million light in terms of just temporarily reduced expenses on consultants, travel and client entertainment, etc. So if you would -- thinking about normalizing, it's kind of that 84 number that we've talked about in the past. That's probably a good number to use.
Michael Collins -- Chairman and Chief Executive Officer
Yeah. And Mike, I'd also say, just the broader expense outlook. We're still focused completely on longer term, moving on client-facing positions as we restructure Bermuda and Cayman to Halifax. So we currently have about 150 employees in Halifax. It's worked really quite well even during the pandemic. So that's a great office and we're going to continue to build it. So that's longer term. Unfortunately, obviously, during this period, it's very difficult to actually execute some of those plans, but longer term, that's still going to be our focus and it will continue to be a bigger office.
Shorter term, near-term, much more tactical. So obviously, Michael has mentioned, we were not traveling, no entertainment, so that actually helps a bit. We don't think that's going to come back to where it was before, so we'll continue to save money there. Temps and contractors were all over that in all of our jurisdictions. When things are going well, you tend to add quite a bit of project to work, and that's temps and contractors, so we're reducing that. And then structure and then unfortunately headcount across all our jurisdictions. We're assuming we're in a zero interest rate environment going forward. And I don't think any organization can continue with the model they've practiced before. And we need to figure out how to operate more efficiently. so longer term, Halifax; shorter term or near-term, much more tactical, which we'll be talking to you about.
Michael Perito -- KBW -- Analyst
Great. Helpful guys. Thank you. And then maybe switching over to the non-interest income side. Michael Collins, you mentioned some fee -- you're trying to expand some fee opportunities. Any additional color you can provide us there of what some of the opportunities you're looking at that might be material moving forward?
Michael Collins -- Chairman and Chief Executive Officer
Sure. So I think our focus continues to be in-market lending. So I think what we're pretty happy about is coming into the crisis with a healthy balance sheet, $13 billion balance sheet, only $5 billion of loans, all in-market, investment portfolio is predominantly agencies, so no real credit risk there. $180 million in unrealized gains. So we're coming in in a good place. And so I think what that's taught us is what we've been doing the last five years to 10 years is really avoiding out-of-market lending.
So the opportunities, other than acquisitions, we're thinking about are in the Channel Islands. So the acquisition of ABN gave us about $500 million of sort of private banking and small managed company corporate loans, and that's kind of went in our appetite. And looking at the Channel Islands, AA- jurisdiction, predominantly financial services, not really sort of focused on tourism. Great opportunity I think to make, particularly Guernsey and possibly Jersey look a little bit more like Bermuda and Cayman to be full service banks, sort of high-end residential mortgages potentially. So we're still working through that, but we think there is an opportunity to do in-market lending there as well and actually broaden out the product set to look more like Bermuda and Cayman. So that's what I was referring to.
Michael Perito -- KBW -- Analyst
Great, helpful. Thanks. And then just lastly. I mean, you mentioned that the mortgage deferral was in an effort to kind of bolster the balance sheet of your customer base. I guess, did you -- did you have the desired effect that you guys -- do you think your customers today are feeling OK about kind of their cash position and the outlook or do you think the tourism uncertainty, particularly in Bermuda, but I guess in Cayman as well is still weighing on kind of the customer confidence and business outlook for many of your constituents there on the island?
Michael Schrum -- Group Chief Financial Officer
Yeah. Mike, it's Michael Schrum. I would say, I mean, it's a tough one. Obviously, we're collecting. We're pleased to see the participation rate drops, obviously for the second one. And again, a slight differentiation between probably Cayman and Bermuda where Bermuda had a slightly higher participation rate in the second round than Cayman. I think Bermuda has been a bit slower to get back to work generally, if you look at government statistics for unemployment, etc. So I think it's had the desired effect. It was nice to see the participation rate drop and a number of people saying, this is fine, I can stop paying again. But I have no doubt that there will be a need for us to assist customers even after the six months. And there will be some that perhaps are going to miss going into the winter season in Bermuda, in particular miss their ability to service their loans and we'll have to work with those customers on different solutions. I mean, we still have the playbooks from the last financial crisis in terms of TDRs, etc. So I think we know what to do. It's a little -- still a little bit uncertain, but it's good to see the progress quarter-over-quarter.
Michael Perito -- KBW -- Analyst
Okay, great. Listen, thank you guys. I appreciate you taking the time. Stay well.
Michael Collins -- Chairman and Chief Executive Officer
Thanks, Mike.
Operator
Our next question comes from Alex Twerdahl with Piper Sandler.
Alexander Twerdahl -- Piper Sandler -- Analyst
Hey, good morning, guys.
Michael Collins -- Chairman and Chief Executive Officer
Good morning, Alex.
Alexander Twerdahl -- Piper Sandler -- Analyst
First off, I was wondering if you could just help me understand or help us understand a little bit more into the types of loans that migrated into NPL during the quarter. I think you mentioned one commercial loan and a couple of residential loans. But maybe help us figure out sort of what the potential loss content could be from those types of NPLs and put that in context of what you did with the reserve?
Michael Collins -- Chairman and Chief Executive Officer
Okay. I'll start off and hand it over to Michael. Overall, obviously, we had about $20 million or so roughly go into NAL, and it's a pretty simple story. It's half of it's basically one commercial facility that was really sort of more of a private banking type structured loan that has lots of collateral, but unfortunately has various entities who are sort of going after that collateral. So it's overcollateralized. There is a situation that we need to work through. But we still think that there is plenty of collateral there. But that's the one commercial loan.
The other half of the $20 million or so was a handful of residential mortgages. And just to start off, I would say we do expect residential mortgages in terms of 30-day delinquencies and 90-day NALs to increase. I mean, obviously, we're going into the winter season after not having a tourism season, particularly in Bermuda. So we think that weakness will definitely be in Bermuda and not so much Cayman and not the Channel Island.
So we do want to put that out there. But I would say, because we have this deferral program, which is really almost automatic for the Q2, we typically have a -- usually have a high cure rate. So we have a good team that basically dials for dollars and calls people. It's a small community. We know people. And it's really trying to get -- convince them to pay us versus other creditors potentially, and we usually do a good job. During Q2, during the lockdown, obviously, we weren't doing that. It would be inappropriate to be calling people at home when they can't even be working.
So I think some of the mortgage non-accruals, most of those facilities were already having trouble sort of 30-day delinquencies. And I think they used that period where no one else was paying to sort of say, OK, I won't pay this month and that was during basically the lockdown. So we think that was partially driven by that. But I would say, we have to look at the economy. Bermuda is shrinking. And we do think we're going to have some residential mortgage issues. But the great news in our book is we're 60% residential mortgages.
So again, coming into the crisis, we don't have a lot of hospitality tourism loans. The hotel loans are really well underwritten, only $7 million of restaurant loans. And the commercial loans are all office buildings supported by reinsurance company leases. So it's a good problem to have. I think it's a residential mortgage problem. It's granular, but that will tick up we think.
Alexander Twerdahl -- Piper Sandler -- Analyst
And can you remind us to LTVs?
Michael Schrum -- Group Chief Financial Officer
Sorry.
Alexander Twerdahl -- Piper Sandler -- Analyst
Sorry, Michael. Go ahead.
Michael Schrum -- Group Chief Financial Officer
So yeah, I think we just talked on it in the remarks. So I'll just point you like to Note 6, you can kind of see that migration that we talked about. It was around 50-50. So one C&I loan was 50% of that. Michael, talked about that in detail. It's just collateral dispute that's caught up in the courts really. But we feel pretty good about it. All the actors are acting in the right way. The collateral is there. It's just going to kind of work its way through.
On the resi side, really folks were already kind of 30 days past due. And you can see that in Note 6 if you look at the rating migration down the track there. We're obviously not expecting anything to come through next quarter just because of the deferral program. So ultimately, as Michael said, in Q4 will be kind of when we know the full picture. Obviously during the opt-in we've collected notes from people that we've called and talked to during opt-in. And so far, most people look ready to come back and pay their mortgages. So some uncertainty, but not a big concern I would say.
Alexander Twerdahl -- Piper Sandler -- Analyst
Okay. And then the residential loans you're talking about that are kind of in the at-risk bucket, can you remind us LTVs and etc?
Michael Schrum -- Group Chief Financial Officer
So I mean, generally, I think 70% of our mortgages are below 70% LTV. I think that's the right number. We've disclosed this in the past. But the overall book [Phonetic] is well seasoned in the high-50s, I believe in Bermuda. So there is also capacity, but it all depends on the employment picture and getting people back to work. There is very little in the -- that has an LTV above 90%.
Alexander Twerdahl -- Piper Sandler -- Analyst
Okay. And then just a couple of clarification questions. When you're talking about the sort of desire to rejigger the efforts in the Channel Islands and grow loans there. Can you kind of talk a little bit about sort of the appetite to bring the loan to deposit ratio for the overall bank higher? If that's something that we should expect in the next couple of years as well as kind of the currency implications for funding loans in the Channel Islands? Are those going to be sterling-denominated, euro-denominated? And kind of how should we think about the sort of capacity on the balance sheet to actually fund the loan growth that we could potentially hope for?
Michael Collins -- Chairman and Chief Executive Officer
Well, I think overall, I think we would still continue with our statement that is not a loan growth story. It's going to spike up -- loans will spike up a bit because we're doing some sovereign lending in Cayman, Guernsey and Jersey. So you'll see a bit of that. The initiative in the Channel Islands in terms of high-end private banking residential mortgages is organic. So we hire a team. We understand the market. We started slowly. So that will ramp up over time. I would just look at it. So basically the London mortgages, which is about $1 billion sterling in Central London now are funded from sterling deposits in Guernsey and Jersey. And this is just another in-market lending opportunity for those two banks using their sterling deposits.
Michael Schrum -- Group Chief Financial Officer
Yeah. So we don't anticipate needing any kind of derivatives or anything. There is organic funding from the ABN acquisition in sterling. As you know, we got both dollars and sterling. We've run off pretty much to euros now. So we believe with that organic growth profile over time that we'll just actually activate the deposits, because there is no secondary asset markets that are particularly attractive in the current rate environment.
Alexander Twerdahl -- Piper Sandler -- Analyst
Okay, understood. And then just another quick...
Michael Schrum -- Group Chief Financial Officer
[Speech Overlap] ratio. So I wouldn't expect for that to materially move up because we're also obviously hoping that and planning to grow deposit levels with existing clients both in Jersey and Guernsey, following the Deutsche Bank and the ABN acquisition.
Alexander Twerdahl -- Piper Sandler -- Analyst
Okay. And then just a final clarification on expenses. You said, kind of 84 is sort of the normalized with $2 million or so light due to the COVID lockdown. I mean, would it be fair to assume that the third quarter is going to be another light quarter for expenses just given -- I can't imagine a lot of the traveling that normally would have taken place is still not going to take place in the third quarter?
Michael Schrum -- Group Chief Financial Officer
Yeah. No, I would definitely expect that, Alex. So I would expect it light. But again, some of it is temporary. We are relationship-driven bank. So we want to see our customers. We want to talk to them about their needs. So it's not kind of an AI leverage play, it's a relationship-driven model. And so some of that will come back over time. But I think you're right, temporarily that will continue to be under those side.
Alexander Twerdahl -- Piper Sandler -- Analyst
Thanks for taking my questions.
Michael Collins -- Chairman and Chief Executive Officer
Thanks, Alex.
Operator
[Operator Instructions] Our next question will come from Timur Braziler with Wells Fargo Securities.
Timur Braziler -- Wells Fargo Securities -- Analyst
Hi, good morning.
Michael Collins -- Chairman and Chief Executive Officer
Hi, Timur.
Timur Braziler -- Wells Fargo Securities -- Analyst
Maybe just circling back to the mortgages that migrated to non-performing this quarter, I guess how does these compare to the remainder of the mortgage portfolio? I guess, what type of deterioration that you saw that weren't the migration to non-performing? And also the geography of these mortgages would be great?
Michael Collins -- Chairman and Chief Executive Officer
So this is -- these are in Bermuda. So I mean, I would say, Cayman is less than 1%, 30-day delinquencies right now. So a different environment. So all of these mortgages were in Bermuda. And coming into Q2 when we did kind of the automatic deferral, most of these mortgages were already struggling with 30-day delinquencies. It's a pool of mortgages that's kind of in and out of 30-day delinquencies. So we know who they are. They're not 90 days. They get 45 days. And then they get some cash and they pay it. So not really bad borrowers, but clearly struggling off and on with one or two people working. So that's really that group right there.
And so I think when we did the automatic deferral, I think it gave them an opportunity to say, OK, well, no one else is paying then I'm not paying. We didn't allow them that automatic deferral because we put them out because they were 30-day delinquent. So they already had issues. That's not to say other parts of the book aren't going to start to struggle, but that was kind of this group.
Michael Schrum -- Group Chief Financial Officer
Yeah. I mean, I would say they would be the ones over the last six quarters that have been in and out, right, of 30-day. So if you think about the risk buckets from an LTV perspective, they're reflective of the rest of the book. But from a DS office -- but from a servicing perspective, they've been struggling through variety of reasons, whether it's unemployment or life events, divorce, etc. So there was a need for us to find a structured solution for those. But I wouldn't say from a debt service capacity perspective that they were at the higher end of the risk spectrum.
Michael Collins -- Chairman and Chief Executive Officer
And Timur, the reason they generally didn't go into 90-day non-accrual in the past is because we were all over them. So we called them, we called them, and it's very high cure rates. So we just weren't able to do that obviously during the lockdowns.
Timur Braziler -- Wells Fargo Securities -- Analyst
Okay. That makes sense. And then just looking more broadly at -- we can stick with the Bermuda residential portfolio. I mean, there can't be much clarity into what's going on there until we start getting broader reopening. Is that a fair statement? And I guess, pairing that with the current allowance ratio, is the thought internally that that's likely to continue moving higher, albeit, probably at a slower pace than we saw in the first two quarters?
Michael Schrum -- Group Chief Financial Officer
Yeah. It's a great question. So there is uncertainty remaining until Q4, obviously, it's when we'll know for sure. I mean, we're not expecting, obviously, anything really in the very near-term just because of the opt-in. But again, anecdotally, when we call clients, there is a significant progress in terms of people being willing and able to pay for the second round, which is good news. And then I mean, the only other comparison I really have is the sort of 2013, '14 where we had about 50 or 60 TDRs come through, through the pipe, and most of them are still performing, obviously, today. But yeah, I mean clarity wise, I think it's going to be a couple of quarters. And it will primarily depend on the speed at which people get back to work and how many people get back to work because that's going to ultimately determine whether -- their ability to pay.
Michael Collins -- Chairman and Chief Executive Officer
I think that's a good way to describe it, Timur, that we think it's going to continue to tick-up, but probably at somewhat of a slower pace because these were the troubled mortgages, to start with. But it's going to be a long winter in Bermuda. The good news for Cayman is that they're keeping their airport close till September. And so they're going to open right up into their winter season, which is their high season. Bermuda's timing is a little less fortunate. I mean, we have daily Delta flights, we've got a BA flight coming in. But you get 60 to 100 people a day is not really tourism. So I think going into the winter, it's going to be -- the issues will be in Bermuda, not -- certainly not in Cayman and not in the Channel Islands.
Timur Braziler -- Wells Fargo Securities -- Analyst
Okay, that's helpful. Thank you. And then switching over to margin. Quite impressive to see the magnitude of decline for deposit costs, certainly that benefited NIM this quarter. But as we look out, I guess, how should we be thinking about incremental pressure on asset yields? And it looks like at this point much of that is going to flow through the margin. Is that the right way of thinking about it or is there incremental room on the deposits?
Michael Schrum -- Group Chief Financial Officer
So I mean, if I could start off just on the Bermuda resi, obviously, the 50 basis point decline was effective on the 1st of May. So there is a bit more pressure coming through on the resi side next quarter, call it, couple of handful of basis points on the resi side just from a full quarter impact. Obviously, the investment portfolio will depend on where prepayment speeds go. At the moment, we don't have a lot of premium amortization in that book. But prepayment speeds are probably in the high-teens coming into the second quarter, which is up quite materially. So we're getting a lot more maturities coming through with lower reinvestment rates, which is driving duration. But if you think about that from an annual -- annualized basis and you think about $180 million positive mark-to-market, which is going to come through the asset yield on the investment side, that kind of blends back to 20 basis points, 30 basis points a year as long as interest rates are where they are today. So I think we've seen most of the effects on the loan asset yield come through this quarter.
On the deposit side, for the first quarter, our interest-bearing demand deposits were actually yielding negative, partly because the balance is strong and partly because the euros were repriced down. So we had negative 12 basis points actually on that, which drove down the average cost of deposits. And then the rollovers are going to come through in a three month, six month bucket. So there is a bit more benefit to come through in the short-term as well, while those rollovers reprice into lower rates.
So I think overall, you could see five basis points to 15 basis points in the near-term and then it should stabilize. And as we then start to ladder out the ABN deposits into investment securities now that we have a year's worth of data, we should see some pick up on the overall NIM, about -- probably about 40 basis points and we'll probably ladder that out $100 million a quarter for the next six quarters or so. That will help a little bit, again, offset some of the compression in near-term and then it should kind of start to turn a little bit.
Timur Braziler -- Wells Fargo Securities -- Analyst
Okay. And then just last one for me. Looking at the ABN deposits, and I think there is a slide on it, but maybe you can just tell us the numbers that would be easier. But what's the remaining component that's euro-based that you're expecting to move off the balance sheet in the near-term?
Michael Schrum -- Group Chief Financial Officer
So we have about a couple of hundred million left of that ABN book. And it's -- I'm not sure it's going to move, but it's been repriced to market. So we're OK keeping it as part of a broader relationship. Some of the funds have sterling, dollars and euros, so there are different tranches of their investment proposition to clients, and obviously they prefer to deal with one banking institution. As long as we are pricing them to market and we can cover our capital costs on the euros, we get the whole relationship, that's OK. So I'm not expecting a lot more attrition.
Timur Braziler -- Wells Fargo Securities -- Analyst
Great. Thank you very much.
Michael Collins -- Chairman and Chief Executive Officer
Thanks, Timur.
Operator
This concludes our question-and-answer session. And I would like to hand the call back over to management for any closing remarks.
Michael Collins -- Chairman and Chief Executive Officer
Thank you, Isley, and thanks everyone for dialing in today. We know it's busy day for calls. We look forward to speaking with you again next quarter. Have a great day.
Operator
[Operator Closing Remarks]
Duration: 42 minutes
Call participants:
Noah Fields -- Head of Investor Relations
Michael Collins -- Chairman and Chief Executive Officer
Michael Schrum -- Group Chief Financial Officer
Michael Perito -- KBW -- Analyst
Alexander Twerdahl -- Piper Sandler -- Analyst
Timur Braziler -- Wells Fargo Securities -- Analyst
