Image source: The Motley Fool.
First Hawaiian, Inc. (FHB 3.19%)
Q2Â 2020 Earnings Call
Jul 24, 2020, 1:00 p.m. ET
Contents:
- Prepared Remarks
- Questions and Answers
- Call Participants
Prepared Remarks:
Operator
Ladies and gentlemen, thank you for standing by and welcome to the First Hawaiian Second Quarter 2020 Earnings Conference Call. [Operator instructions]
I would now like to hand the call over to Kevin Haseyama. Please go ahead.
Kevin Haseyama -- Strategic Planning and Investor Relations Manager
Thank you, Michelle, and thank you everyone for joining us as we review our financial results for the second quarter of 2020. With me today are Bob Harrison, Chairman, President and CEO; Ravi Mallela, CFO and Ralph Mesick, Chief Risk Officer.
We have prepared a slide presentation that we will refer to in our remarks today. The presentation is available for downloading and viewing on our website at fhb.com in the Investor Relations section.
During today's call, we will be making forward-looking statements. So please refer to Slide 1 for our Safe Harbor statement. We will also discuss certain non-GAAP measures. The appendix to this presentation contains reconciliations of these non-GAAP financial measurements to the most comparably -- directly comparable GAAP measurements.
And now I'll turn the call over to Bob.
Robert Harrison -- Chairman, President and Chief Executive Officer
Thank you, Kevin. Good morning, everyone. Thank you for joining us. I'd like to start with updates on the current situation here in Hawaii, then we'll update you on deferrals, credit and provisioning before we go into our normal review of the quarter.
The economy within Hawaii has reopened. The quarantine requirements for airline travel was lifted in June. And while our number of cases has risen following the reopening of the local economy, we are still doing a good job of controlling the spread of the virus. The trailing 14-day average of new cases is 26 as of yesterday. With the reopening of the local economy, unemployment fell to 13.9% in June, down from over 23% in May. However, the quarantine requirement for transpacific travel has been extended to August 31st.
During the quarter, the Bank leveraged off our digital investments to quickly respond and develop an online application for the PPP program. This allowed us to approve 6,000 loans for over $940 million. And at a time when many people are being very cautious about unnecessary contact, we also launched our contactless debit and credit cards. We've also begun reopening our branches and bringing some employees back to the office while incorporating work from home as part of our normal business model. We're continuing to support the community and we've launched the Aloha for Hawaii Fund to support the restaurant industry and donated over $1 million to agencies providing COVID relief. We also donated $1 million to assist some recent high school graduates as they transition to college or vocational school.
Moving to Slide 3, I'd like to go with -- provide you an update on our loan deferrals. As you recall, our strategy as a relationship bank was to be very proactive in working with borrowers who faced a liquidity shock as the economy went into a shutdown. We did this by providing customers with payment deferrals to allow them to adjust to the changing circumstances. Our expectation at that time was that many would adapt to a new normal and we expected most of our customers to return to regular payments. Our experience so far has borne this out and we're seeing the majority of customers returning to payment or indicating they will be returning to payment.
In the commercial segment when we followed up with each individual customer who went on deferral, we also reassessed their financial situation to update our risk rating on loans. Currently over 95% of the commercial customers who indicated that they will not -- have indicated they will not require any additional relief and have either returned to payment or intend to return to payment at the end of the deferral period.
Following the review process, about 14% of the commercial portfolio was downgraded to criticized levels for those that took a deferral. Our visibility into the consumer space is not quite as clear as the commercial space, but early indications are good. For the non-mortgage consumer portfolio, deferrals were up to 90 days, the majority of which end in June and July. What we're seeing is that of the approximately $153 million of consumer loans that came off deferral, over 90% returned to payment. Of the remaining $126 million still on deferral, $41 million is paying.
Residential mortgage borrowers were provided deferrals of up to six months following the program setup by the GSEs, so those borrowers were still in their deferral periods. When you put all of that together for the current situation, of the original $3 billion of loans that went on deferral, over $2.2 billion or 73% have returned to payment or said they will return to payment at the end of their deferral period. Residential mortgages on six month deferrals make up the majority of the remaining $814 million of loans, still on deferral or needing relief. Excluding residential mortgages, a little over $240 million or 2% of total loans remain on deferral or are seeking additional relief.
Turning to Slide 4, you can see a recap of the selected industry slide that we presented last quarter. For this quarter, we also included our auto dealer exposure on the table since this was an area of interest and we don't break it out elsewhere. As you can see on the slide, other than PPP loans, there have been no meaningful changes in our exposures to these industries. On the whole, deferrals and PPP loans helped many of our clients build a liquidity base to help them through a challenging time. With that, I'll turn it over to Ralph to go over the credit trends.
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Thank you, Bob. Slide 5 provides highlights on asset quality this quarter. During the quarter, we updated our outlook for the economy and completed a comprehensive review on the commercial loan portfolio. The review focused on customers who took deferrals, those in industries that were impacted by the shutdown, and larger exposures regardless of payment status or industry. These activities led to reclassifications and increase in provisioning, and writedowns to a few credits that were already classified prior to the onset of the pandemic.
For the quarter, the provision was $55.4 million, up from $41.2 million in Q1, and net charge-offs increased by [Phonetic] $23.4 million, up from $6.1 million in the prior quarter. The higher provisioning reflected the expected impacts of a weaker economy and material increase in the level of criticized commercial loans. Criticized commercial loans increased by 135% to $742.4 million at quarter-end.
The net charge-offs included approximately $16 million in writedowns on several loans. These loans were classified prior to the shutdown and subsequently placed on non-accrual during the credit review process. Loans that were performing or 90 days past due increased by $25.1 million to $42.9 million at quarter-end. The increase was fully attributable to the loans that were written down. Past due loans fell $19.5 million quarter-over-quarter to $9.1 million.
Slide 6 shows the composition of our commercial portfolio by risk rating as of quarter-end. The portfolio review conducted in the second quarter assessed individual borrower risk by considering a company's financial profile, taking a forward view that assumed a prolonged period of stress. As a result of that effort, we downgraded almost 6% of the loans. Special mention loans increased by $363.4 million to $576.5 million and substandard loans increased by $63.5 million to $165.9 million. Re-rating of the commercial loans helped us refine our estimate of future credit costs and identify higher risk credits needing more active management.
Slide 7 provides a recap of the allowance for the quarter by disclosure segments. In Q1, we embedded a large overlay into the loss estimate. This was necessary given the lack of information that was available at that time and with an updated economic outlook and the regrading of the commercial loan portfolio, more of the provisions shifted into the reserve model and away from the overlay. As shown on the table, roughly $49.5 million of the provision expense taken was allocated to the allowance with the largest build in the consumer, C&I and CRE segments.
The building consumer loans was largely due to an increase in our economic modifier based on our forecasts over the next 12 months. The increase in C&I and CRE areas reflected the change in the modifier as well as downgrades of credits within those portfolios. With the increased provision, the allowance now represents 1.4% of total balances, including PPP loans, and 1.5% net of the PPP loans.
And with that, I'll now turn the call back over to Bob.
Robert Harrison -- Chairman, President and Chief Executive Officer
Thank you, Ralph. If you turn to Slide 8, you see the second quarter highlights. Earnings were $20 million for the quarter and our pre-tax pre-provision net revenue was solid at $82 million. We did take a $55.4 million provision for credit losses due to the evolving impact of the pandemic as Ralph just reviewed, but we had strong deposit growth in the quarter and our cost of deposits declined by 50% to 19 basis points. We finished the quarter with strong capital and liquidity.
Turning to Slide 9, we remain well capitalized and our common equity Tier 1 ratio was 11.86% at the end of the quarter. Our CET1 ratio increased by 21 basis points in the quarter primarily driven by a shift in asset mix. Based on our current economic outlook, we expect to generate sufficient earnings to remain well capitalized, support future loan growth, and sustain the current dividend. If economic conditions become significantly worse than we anticipate, we'll reassess our ability to maintain the dividend at existing levels.
Now, I'll turn it over to Ravi to go over the rest of the balance sheet and income statement.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Thank you, Bob. Turning to Slide 10, period-end loans and leases were $13.8 billion, up $384 million or 2.9% versus the prior quarter. PPP balances at the end of the quarter were $916 million. Excluding PPP loans, loan balances declined $532 million or 3.9% for the quarter. Shared national credit declined $208 million, dealer flooring declined approximately $180 million and consumer loans declined $76 million. The rest of the decline in loan balances came from loans in the commercial portfolio. Looking forward, we expect that loan growth will continue to be challenging.
Turning to Slide 11. Total deposit balances ended the quarter at $19.4 billion, a $2.3 billion increase versus the prior quarter. Core consumer and commercial deposit balances grew by $1.4 billion. We estimate that the PPP loan proceeds contributed to about $500 million of that growth. Total public deposits increased by $820 million in the second quarter. This included $609 million increase in public time deposits. Our cost of deposits fell by 19 basis points or 50% to 19 basis points in the second quarter.
Turning to Slide 12. Net interest income in the second quarter was $127.8 million, a $10.9 million decrease versus the prior quarter. That decrease was primarily due to lower yields on loans and investment securities, partially offset by lower rates on deposits and higher loan and investment security balances. Net interest margin in the second quarter was 2.58%, a 54 basis point decrease from the prior quarter. The decline in margin was primarily due to the 150 basis points of rate cuts in the first quarter and the higher cash levels have been carried during the quarter. Average cash balances were approximately $920 million higher in Q2 and were elevated by strong growth in commercial and consumer deposits, deposits generated from PPP loan balances and an increase in public deposits.
During the quarter, as we got better visibility into the PPP loan funding and deposit dynamics, we deployed excess cash into the securities portfolio. Over the next two quarters, we expect cash levels to decline as we pay down $200 million in FHLB advances that mature in late July, reduced public time deposits while remaining PPP balances get used. During the second quarter, we began deploying excess cash into the investment portfolio which will be accretive to interest income in the third quarter, but will pressure the margin. As a result we conservatively estimate that the margin will increase to the 2.65% to 2.70% range in the third quarter barring any significant PPP forgiveness.
Turning to Slide 13, noninterest income was $45.7 million, $3.6 million lower than the prior quarter. The reduction in non-interest income in Q2 was primarily driven by lower customer activity as a result of the COVID-19 restrictions. Noninterest expenses in Q2 were $91.5 million, about $5 million lower than in the previous quarter. The lower noninterest expenses were primarily due to higher deferred loan costs related to the origination of PPP loans and lower cards rewards expenses due to the lower card activity.
And now I will turn it back to Bob.
Robert Harrison -- Chairman, President and Chief Executive Officer
Thank you, Ravi. So to wrap it up, at the state level we reopened the local economy and indications are that we're doing a good job controlling the virus even as more people are returning to their everyday activities. However, given the COVID situation on the mainland and the additional push back of reopening the local economy to tourism, there remains a fair amount of uncertainty when we will be able to welcome back tourists. Having said that, what is a [Phonetic] good job of controlling the virus, this really makes us an attractive place to visit when that's available. During this period of uncertainty, we will remain a source of strength for our customers, our employees and the community.
And with that we have to take any of your questions.
Questions and Answers:
Operator
[Operator Instructions] Our first question comes from Ebrahim Poonawala of Bank of America Securities. Your line is open.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Hey. Good morning.
Robert Harrison -- Chairman, President and Chief Executive Officer
Good morning, Ebrahim.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Good morning, Ebrahim.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
I guess, just first, Ravi, if I could follow up with you on your margin guidance of 2.65% to 2.70% in the third quarter, sorry if I missed it, but does that exclude everything PPP and is that kind of a base that you think where margin becomes relatively defensible or stabilizes in that range as we think beyond third quarter and maybe into -- well into the first half of next year? Understanding that you're not guiding that out, but I'm just wondering is margin stabilizing in that range. Is that sort of a reasonable way to think about it?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
We gave that guidance for Q3, that 2.65% to 2.70%. Certainly, we expect the NIM to recover, but there are many headwinds along the way here. We should be able to reduce cash levels over the course of one to two quarters. We have about $300 million in maturities on the public time side every quarter that will roll off. We're going to be paying off $200 million in FHLB borrowings at the end of the month. And we've also started to see cash levels decline as stimulus gets deployed.
But there are headwinds we face. We talked about the -- we could see additional deposit flows into accounts. We had a very strong deposit growth in the quarter, can't forget that. And as PPP loans get used, we could see some of that come back to the balance sheet. So there's a lot of moving parts here for the future, but we expect NIM, at least in Q3, to return to that 2.65% to 2.70% range.
Robert Harrison -- Chairman, President and Chief Executive Officer
And just to add to that, Ebrahim, this is Bob, that we haven't forecast in any of the forgiveness for PPP. We're just assuming it runs its course over the two-year period because it's just too hard to guess of when that's going to happen. Rather than estimate it, we're just letting it run for the two-year period in our modeling.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Got it. So that's helpful. And just separately, Bob, I guess one around the reopening of tourism like, talk to us in terms of what's baked into your reserving at the end of the quarter around things reopening back up? How soon do you expect that in your forecast? And the longer this takes and let's say if it gets pushed out for another three months, what does that mean for your reserving and credit outlook and just your resiliency of the customers to kind of live through this and then still be sort of -- be able to service their debt on the other side?
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah. Now, that's an excellent question. Let me just start and then I'll hand it over to Rob because that really ties into how we looked at our economic model for CECL. And we take a pretty conservative view on things as you've seen and watch us over the last several years and that ties in with how our outlook is for the economy.
But, Ralph, do you want to kind of go through it in more detail?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Yeah. In terms of the CECL forecast, we basically looked at economic forecast and assumed a pretty severe recession. I think we estimated the state effective unemployment would be in the low to mid-teens. Personal income would drop in the upper single digits and probably we'd see no meaningful return in tourism until later in the year.
So looking at visitor arrivals down 60% and then state GDP dropping probably low mid-teens, which is probably twice the national rate. And in our model, I think the local unemployment rate is probably the strongest correlation that we have to sort of our base loss model.
Robert Harrison -- Chairman, President and Chief Executive Officer
So, to wrap that up, we're not forecasting a significant recovery at all through the end of the year.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Understood. And just the loans which were sort of -- the 14% you mentioned, deferred loans that went into criticized, give us some color on the borrower base and what happens with those loans, like do they move to kind of the default and workout process, your sense around any loss severity, if you could share any of that with us.
Robert Harrison -- Chairman, President and Chief Executive Officer
We can't talk about individual loans, of course, but I will say they were very much in the tourism-related industries, and we have been working with them, and they have been making progress. But, Ralph, maybe you want to add some comments.
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Yeah. So when we did the revalidation exercise, we asked the loan officers to take a 12-month forward view. We gave them a very severe -- a very stressed environment to sort of look at that view. And if you look at the borrowers business, think about the impact that COVID would have, and what their liquidity was to kind of get through a very long period of reduced activity. So a lot of the credits that were downgraded were in sort of the special mention area.
So these are areas where we sort of say there's potential weakness, not a well-defined weakness. And that potential weakness is really around kind of this prospective view. So we wanted to really be able to understand which clients we had to focus on. And then now going forward, we're going to be looking at each of those clients and looking at the assumptions that we've made in terms of how they would perform over time and sort of updating that.
So if people do better than anticipated or can then we would probably upgrade those credits. And if not, we will look at further if there would be a need to downgrade. And if you can see -- if you see the -- what we have right now in terms of the composition, the bulk of the downgrades were in that special mention category.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Well, thanks for taking my questions.
Operator
Our next question comes from Steven Alexopoulus of J.P. Morgan. Your line is open.
Steven Alexopoulus -- J.P. Morgan -- Analyst
Hi, everybody.
Robert Harrison -- Chairman, President and Chief Executive Officer
Hi, Steve.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Hi, Steve.
Steven Alexopoulus -- J.P. Morgan -- Analyst
I wanted to first start with the dividend which you didn't cover with earnings and, Bob, I heard your commentary. You think you'll cover it -- you'll have sufficient earnings to cover it moving forward. But can you talk about your comfort level in maintaining an elevated payout ratio and maybe where you'd like to see that move to over time?
Robert Harrison -- Chairman, President and Chief Executive Officer
Excellent question. I mean, we're now in a period of additional stress and the last two quarters have resulted in substantial increase in provisioning. What we have always said in the past is looking at a 50% payout ratio, and we think that as the economy normalizes over the next couple of quarters we'll get back to that payout ratio.
It might take a little bit of time, but we're optimistic on Hawaii long term, and we're just going to keep working through this. We have taken conservative marks as Ralph mentioned, and we think that positions us well for a weaker economy in the near term.
Steven Alexopoulus -- J.P. Morgan -- Analyst
Got you. Okay. That's helpful. And then when we look at the slide that you're calling out the select industries and we look at the auto related where the deferrals are pretty high, can you give us some color on what you're seeing there? I assume that's all floor plan in terms of the C&I bucket. But maybe give some color there.
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah. That's correct. The vast majority of the C&I bucket is floor plan. And as we had talked about in the first quarter, virtually all of our dealer customers, you can see 85% of them, took payment deferrals. And all but maybe one very small customer came back to regular payment. And so they are actually doing quite well. One of the issues in our loan portfolio is they paid down their flooring line and they're selling all of the cars and have had difficulty getting new cars from the factory. So they've done quite well coming out of the pandemic.
Steven Alexopoulus -- J.P. Morgan -- Analyst
That's helpful. And then one of the areas which has always been a focus for you guys is the shared national credits. And I know you've exited some of those over the past few quarters. I'm just curious how are those performing here compared to the rest of the portfolio?
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah, what we saw and if you see in the hospitality hotel area, a number the draws in the shared national credit portfolio right before quarter-end and into early April, about $0.25 million. And virtually all of that got paid off in May and June. So they use the lines as expected. And as they relooked at their cash needs or went to the capital markets, they paid off those draws. So, we're seeing it used exactly as we expected. We're not seeing undue stress in that portfolio. Although having said that, that is the larger loans that we did look at every single one of those in our analysis.
Steven Alexopoulus -- J.P. Morgan -- Analyst
Okay. And then finally, Bob, as we think about the reserve, so the first quarter we had a big adjustment, the overlays in there. This quarter you had the downgrades. As we think about the third quarter, it sounded like you're taking a fairly harsh outlook in terms of the economic assumptions. Should we see a reserve build further in the third quarter? It seems like it should be behind you here. Right?
Robert Harrison -- Chairman, President and Chief Executive Officer
We're very comfortable with the levels of where they're at now, and unless the economic outlook changes significantly, we would foresee that provisioning would come down.
Steven Alexopoulus -- J.P. Morgan -- Analyst
Okay. Okay. Great. Thanks for all the color.
Operator
Our next question comes from Andrew Liesch of Piper Sandler. Your line is open.
Andrew Liesch -- Piper Sandler -- Analyst
Good morning, everyone.
Robert Harrison -- Chairman, President and Chief Executive Officer
Good morning.
Andrew Liesch -- Piper Sandler -- Analyst
Just questions on the deposit growth outside of the PPP funds and the public funds. There's pretty outstanding growth from local consumers and commercial businesses. What was behind that? Anything you can point to?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Well, I think it was a lot of different activities that we're seeing with our commercial and consumer borrower, I mean, our customers. We have a very rational deposit market here in Hawaii and we just saw a lot of build as people moved their money and their funds into deposit accounts.
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah. And to add to those comments, Andrew, Ravi said we also saw some money coming back from our sweep accounts back onto the balance sheet. And that's part of that. So, these are funds that they just moved back. That wasn't a huge amount, but that was certainly part of it. And then separately on the public side, as we've talked about in the past, being the state depository, we did -- the state did receive its CARES money, and we worked closely with them over the course of the quarter to manage that huge influx of funds as it landed in their accounts.
Andrew Liesch -- Piper Sandler -- Analyst
Okay. And then how should we think about the expense base going forward? You had the increased deferral costs and the lower card expense as well and I would also assume that maybe the less travel and entertainment costs. But what should -- where should expenses move from here? I mean, when do you think you're going to see it increase back to normalized run rate?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Yeah. I mean, I think from an expense perspective, maybe let me just walk you through a little bit of the drivers there. In 2020, we're absorbing the last bit of reimbursements that ended about $6.5 million. And we've mentioned on previous calls, we're continuing to invest in technology. We're in the process of implementing a new core platform, and we mentioned about a 1% to 2% increase in expenses, and we're committed to those activities and projects improving our technology and implementing our core. So we'll continue to invest in those areas.
In the areas that we could see opportunities, I think that'll depend on the economic activity that we see in the areas. The two big drivers on expenses this quarter were tied to -- were tied to the PPP loans, deferred loan costs that we absorbed as a reduction of the salaries and expenses. We also saw declines in the use of our cards and card rewards expenses associated with lower activity. So we're looking for opportunities.
We're constantly looking for those opportunities. We're very cost-focused and expense-focused as an organization. But I think a lot of the drivers that will occur on the expense side will be driven by economic activity and expenses associated with those economic activities.
Robert Harrison -- Chairman, President and Chief Executive Officer
And, Andrew, this is Bob. Just to add to those comments, on the other side of the ledger, just a reminder, we did waive all of our ATM fees during the second quarter to help our customers out. And so that will be coming back on on the revenue side in the third quarter.
Andrew Liesch -- Piper Sandler -- Analyst
Understood. Thanks so much. You covered all my other questions.
Operator
Our next question comes from Jacque Bohlen of KBW. Your line is open.
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Hi. Good morning, everyone.
Robert Harrison -- Chairman, President and Chief Executive Officer
Good morning.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Good morning.
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Bob, I wanted to follow up on your comments related to the waived fees in the quarter. Was that the sole driver of the quarterly variance between 1Q and 2Q in terms of service charges and other fee income, or is some of that impacted by customer behavior so it could carry over into third quarter as well?
Robert Harrison -- Chairman, President and Chief Executive Officer
Well, much of the second quarter, we were stay at home, and so that clearly had an impact on customer activity and customer behavior. So you saw fewer credit and debit transactions. You saw the higher loan balances resulted in lower overdraft fees. There were a number of different drivers that affected that. Ravi, did I miss anything?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
I would just add, in terms of economic activity, we also saw lower merchant services fees during the quarter. And I'd say, Jacque, that we saw it bottom out in April and recover a little bit in May and June as we started to open the local -- reopen the local economy.
I think a lot of -- looking forward, a lot of it will depend on how well we'll be able to open the local economy and whether we'll be able to open it broadly. But I think what we saw was stabilization in the middle of the quarter with a little bit of an increase over the course of the quarter as we reopen to ourselves.
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Okay. And then understanding that the other income line item can be very volatile, what was the driver of the increase on a linked-quarter basis?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Yeah. The driver of -- are you talking about non-interest income?
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Yeah.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Yeah. The big drivers on non-interested income was higher BOLI income in the quarter. If you look back at the quarter and think about the quarter, we had a lot of disruptions in the market and some of the underlying investments in our BOLI portfolio were impacted in Q1 and they recovered in Q2. And so you saw about a $2.2 million increase in BOLI Q1 to Q2.
And then we had higher swap fee income in the quarter. And that was just a reflection of some of our customers being able to go in and engage in some swap activity as rates fell to -- and they generated some income in the quarter itself. The offsetting factors there, as Bob and myself mentioned, was just lower service charges on deposit accounts and lower credit and debit card fees and merchant services fees.
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Okay. So looking specifically at the other income line item within noninterest income that went up from about $5 million to about $8 million, was the primary driver of that swap -- higher swap come then?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Yeah. It was about $2.8 million from higher swap fees and about $400,000 in about -- in the miscellaneous categories to get to that $3.2 million.
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Okay. Great. Thank you. That's helpful. And just one quick last one on PPP. So, are you amortizing your expected fees on a quarterly basis?
Robert Harrison -- Chairman, President and Chief Executive Officer
We are. We are. We're taking a pretty conservative view and looking at that over a two-year period.
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Okay. Great. Thank you.
Operator
Our next question comes from Brock Vandervliet of UBS. Your line is open.
Brock Vandervliet -- UBS -- Analyst
Great. Thanks for taking the question. Okay. I covered PPP. I like that disclosure on Slide 7 in terms of the reserve breakdown. A question I had on C&I, we penciled ex PPP, the C&I reserve may be as high as in the 80s basis points. Some of your peers are well above that. Is there something we should look for or you would call out for delta and why that might be lower?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
I think the composition of the portfolio, we do have a lot of SNC credits. They tend to be higher quality even in the downgrading activity that we took. Some of the downgrades were really moving from a very high pass to still a pass grade. So I think that has a lot to do with -- that has a lot to do with the fact that we have a lower reserve in that area.
Brock Vandervliet -- UBS -- Analyst
Okay and as you looked at CECL and the reserve methodology that you went through, how much of a island overlay, if you will, that did you apply to correct for the --Moody's using more of a mainland bent to their numbers, I would assume?
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah. Brock, good question. This is Bob and I'll start and hand it out to Ralph. It's -- we aren't using the Moody's model. We're using the local University of Hawaii Economic Research Organization, the UHERO model is kind of the base model for many of the banks here. It is much more focused on the regional economies, but Ralph, any other comments on that?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Yeah, I mean, 80% of our loan book really is in Hawaii and Guam. So, that's the big driver of our economic modifier to the CECL model.
Brock Vandervliet -- UBS -- Analyst
Okay and lastly I'm sure this is a major topic of parlor conversations, sort of, but once the island is open, any prognostications on take-up from tourist travel?
Robert Harrison -- Chairman, President and Chief Executive Officer
It is going to come down. I think Hawaii is a very attractive place for people to come. We keep hearing that not just anecdotally, but a lot of different sources, but it's going to be driven by people's willingness to get on the airplane. And it seems OK for now, but it's really hard to judge that in volume. So, we'll just have to wait and see.
Brock Vandervliet -- UBS -- Analyst
Okay. Thanks for taking my questions.
Robert Harrison -- Chairman, President and Chief Executive Officer
Thanks, Brock.
Operator
Our next question comes from Jared Shaw of Wells Fargo Securities. Your line is open.
Jared Shaw -- Wells Fargo Securities -- Analyst
Hi, good afternoon. Most of my questions were answered. I guess as you look at the consumer portfolio and especially autos, I mean, as those deferrals have come off, is there any type of adjustment to balance or any type of incremental charge-off you expect to take just to align valuation with -- or balances with current valuation on the cars?
Robert Harrison -- Chairman, President and Chief Executive Officer
Well, in the -- this is Bob. And maybe touch on it and hand it over to Ralph. We baked that into the CECL model and so that's part of our modeling. So, it is part of the model, but Ralph, anything you want to add to that?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Yeah, so in the first quarter when we did the overlay, we did a series of sort of scenarios where we did transitions on that book. And then this quarter, we actually even went further into the book and we looked at loans by the VIN numbers and auction prices. So we added an additional stress to that number in Q2. And now what we're seeing in terms of the return to pay rate in that book, it's better than what we had anticipated. So we're feeling pretty good about the number that we set aside for indirect auto loans.
Jared Shaw -- Wells Fargo Securities -- Analyst
Okay. And then just on the residential portfolio. I guess what are your expectations if -- for performance there if we do see the next UHERO data get incrementally worse. It seems like that's an area of concern in the broader Hawaii discussion base. What's the thoughts around working with customers and the impact of the resi portfolio?
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah, it's something we're certainly paying attention to. I will say that Hawaii has always been and continues to be a supply constrained market and we have underwritten the portfolio conservatively as you remember that. We didn't repeat the slides from last quarter because it didn't change that much, but the average loan to value in the portfolios in the low 60s Ralph [Phonetic]. So, even in a period of significant stress in a supply constrained market it would have to be a very different economic environment where we would be real concerned about the residential portfolio. Ralph, any comments to add to that?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
No, not really, Bob.
Jared Shaw -- Wells Fargo Securities -- Analyst
All right. Thank you.
Operator
Our next question comes from Alex Matters of Goldman Sachs. Your line is open.
Alex Matters -- Goldman Sachs -- Analyst
Hi, good morning. First, just a quick follow-up on the CECL scenario assumptions. Are you factoring in any benefit from the government stimulus programs and deferrals? And if so, are you assuming sort of any continuation of that going forward?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Yeah, in terms of the individual risk ratings of customers, if they did take PPP loans, that probably was factored into their liquidity position. But that's a pretty short term -- that wouldn't really sort of move the needle, so to speak. And we have not sort of assumed any additional programs that become available.
Alex Matters -- Goldman Sachs -- Analyst
Okay. That's helpful. And now on the remaining $800 million or so deferrals sort of still seeking relief, ex the mortgage loans, could you provide some color on what industries those are in and sort of what your plans for workout are there?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
So there's very few in the commercial book, and the way we would approach that is the way you approach any sort of, like, problem credit situation. So each one gets evaluated individually. We come up with a sort of strategy around the loan, action plans and so forth. On the consumer side, we're looking to move from deferrals to modifications, and we've sort of set up a program to deal with a large volume of modifications if necessary. So that's really what we're looking to do, moving from the deferral periods to sort of modification and we think as long as we keep people paying and working with us, that's going to be very helpful for us in terms of getting through this.
Alex Matters -- Goldman Sachs -- Analyst
Okay and then maybe just one more quick one on the mortgage portfolio, could you provide a little color on sort of the timing of when you expect the losses there to eventually come through once the deferrals expire, sort of like how you're approaching that?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Yeah. I mean, that would be something -- it is your typical foreclosure process in the State of Hawaii. You're talking 12 months out from the default, so that would be further down the road. Again, I think we're pretty comfortable with that portfolio right now. We're comfortable with the LTVs. We're comfortable with the location of the collateral. And we have pretty good history, as Bob said, in terms of performance of that portfolio through a cycle given the fact that Hawaii is still supply constrained.
Alex Matters -- Goldman Sachs -- Analyst
Great. Thanks. I actually just have one very quick follow-up. Do you by chance, have the average balance contribution of the PPP loans in the second quarter and what the yield on those was? Thanks.
Robert Harrison -- Chairman, President and Chief Executive Officer
I don't have that off the top, but we'll get back to you with those numbers.
Alex Matters -- Goldman Sachs -- Analyst
Okay. Thanks for taking my questions.
Operator
Our next question comes from Laurie Hunsicker of Compass Point. Your line is open.
Laurie Hunsicker -- Compass Point -- Analyst
Hi, thanks, good morning. With the PPP, how much are you expecting in fees in the $940 million?
Robert Harrison -- Chairman, President and Chief Executive Officer
How much of fees? Yeah, it's right about -- go ahead, Ravi.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
It's about -- it's little over $24 million is what we've estimated.
Laurie Hunsicker -- Compass Point -- Analyst
Okay. And is that net of the $2 million you're donating?
Robert Harrison -- Chairman, President and Chief Executive Officer
We're not connecting the $2 million to the PPP loans.
Laurie Hunsicker -- Compass Point -- Analyst
Got it. Okay. And so --
Robert Harrison -- Chairman, President and Chief Executive Officer
That was out of our foundation.
Laurie Hunsicker -- Compass Point -- Analyst
Okay, OK. So is that showing up then in the other expense line?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
The foundation.
Robert Harrison -- Chairman, President and Chief Executive Officer
Oh, the foundation? No, those are funds that we've put aside over a very long period of time and have been in there. There's a purpose within the foundation that's earning income and that's handled completely separately from the bank.
Laurie Hunsicker -- Compass Point -- Analyst
Okay. Perfect. I didn't know they were just next to each other in the wording of the sum.
Robert Harrison -- Chairman, President and Chief Executive Officer
Oh, sorry about that.
Laurie Hunsicker -- Compass Point -- Analyst
No, no. My confusion. Okay. So just going over to credit. The C&I charge-offs that you have this quarter, the $14 [Phonetic] million, can you share with us what categories those were in?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Yeah, I mean we had four loans that basically were classified pre-COVID when we did the exercise we did this past quarter. We basically sort of changed sort of the outlook for those resolution plans. We did impairment analysis of the charges. We're carrying the balance on the books on non-accrual. And I think we would anticipate that we have paydowns over the next few quarters on those loans as some properties get sold.
Laurie Hunsicker -- Compass Point -- Analyst
And what type of -- I mean what type of C&I loans were those?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
There are a couple of C&I loans and a couple of real estate secured loans.
Robert Harrison -- Chairman, President and Chief Executive Officer
They were tourism related if that's what you're curious -- if that's your question, Laurie. Like.
Laurie Hunsicker -- Compass Point -- Analyst
Sure. I mean were they in the hotel category or the retail category? I mean I think more broadly everything is sort of touching tourism, right, restaurant. I mean I'm just trying to understand what category they broke into.
Robert Harrison -- Chairman, President and Chief Executive Officer
Yeah. We're --
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Given the nature of these situations, we can't really talk a lot about the individual credits. Sorry, Laurie.
Laurie Hunsicker -- Compass Point -- Analyst
Okay. I mean I guess my same question was with the jump in non-performers too. You had both a jump in C&I and a jump in CRE. I just was wanting to understand what categories, not necessarily specific loans but what category your jump in weakness?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
The jump in non-performers were attributed to those four loans.
Laurie Hunsicker -- Compass Point -- Analyst
Oh, so those four loans. Okay. Okay. Just looking at your Slide 4, your hospitality and hotels, only 21% in deferral, round number is $113 [Phonetic] million. That's super, super low compared to what we're seeing with mainland hotels. Can you just, Bob, maybe comment a little bit on the strength there? And then also, even the return to pay within your hotel books, how you're thinking about that. Thanks.
Robert Harrison -- Chairman, President and Chief Executive Officer
Sure. We've always focused on the strength of the sponsor and looked to bank people that have the ability to support themselves through times of stress and I believe that's what we're seeing right now. You know as we look to that -- you know it's a very unusual situation where we have entire hotels still closed for an extended period of time and they have means that they didn't require any support from us. Ralph, anything you would like to add?
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
No, no.
Laurie Hunsicker -- Compass Point -- Analyst
Okay, thanks. And then, I just want to make sure I got this right. So, your dealer floor plan loans right now are $979 million up from $875 million last quarter. Is that correct or am I reading this wrong here? Or maybe a better question is [Speech Overlap].
Robert Harrison -- Chairman, President and Chief Executive Officer
You're reading it off of Slide 4. So, that includes some things that are not floor plan. So, Ravi, do you have the -- Yeah, the dealer floor balances were $795 million at the end of the last quarter. And they're at $615 million this quarter. So, it is down about $180 million.
Laurie Hunsicker -- Compass Point -- Analyst
Dealer floor plan, OK. And then how much of that is California?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
So the mainland this quarter was about $440 million versus $570 million the prior quarter.
Laurie Hunsicker -- Compass Point -- Analyst
Okay. Great. And then do you have the same numbers for your shared national credit book?
Robert Harrison -- Chairman, President and Chief Executive Officer
It's generally about a third in Hawaii and the remainder on the mainland, recalling that we sold -- go ahead, Ravi.
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
The total SNC portfolio this quarter was about $1.1 billion. It was down $208 million from $1.3 billion. The mainland SNC was about $790 million, and of that, $180 million is about $130 million came from the mainland. So that was due to paydowns of loans.
Laurie Hunsicker -- Compass Point -- Analyst
Okay. Great. And then just one last question, following up on Jackie's, do you have the dollar amount of swap income in the June quarter and just the March quarter for comparison, or just even the June quarter [Speech Overlap].
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
I don't have the number for Q1. I can -- we could chat about -- we can give that to you offline. We don't typically get into that level of detail.
Laurie Hunsicker -- Compass Point -- Analyst
Okay. Great. Thanks. I'll leave it there.
Operator
Our next question is a follow-up from Ebrahim Poonawala of Bank of America Securities. Your line is open.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Hey, guys. Thanks for taking my question again. Just wanted to follow-up, Ravi, around the expense outlook, wanted to make sure that I heard -- get the message right. When we look at the benefits that you got this quarter around the cards and the lowest comp expenses, my sense is the card expenses will depend on activities. So if things stay low, that line item will remain low. In terms of the comp, is the 2.4 [Phonetic] does it revert back to what we were in the first quarter? Just how does that play out from here if you can talk to that?
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Yeah, I think it will be -- there will be different drivers. I think what you'll see is maybe a lower deferred comp costs next quarter, but you'll also see continued reductions in salaries expenses driven by lower economic activity. So we've been seeing that to a certain extent in that line and we'll continue to see that as you look out versus Q1.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Got it. And did you say that you expect $300 million in CD public time deposits maturing in each of the next two quarters?
Robert Harrison -- Chairman, President and Chief Executive Officer
We typically see that kind of level of maturity show up on our books on a quarterly basis and it gives us another lever for us to be able to pull depending on what we see in terms of other aspects of the balance sheet.
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Got it. That's all I have. Thank you.
Operator
There are no further questions. I'd like to turn the call back over to Kevin Haseyama for any closing remarks.
Kevin Haseyama -- Strategic Planning and Investor Relations Manager
Thank you, Michelle. We appreciate your interest in First Hawaiian and please feel free to contact me if you have any additional questions. Thanks again for joining us and have a good weekend.
Operator
[Operator Closing Remarks]
Duration: 51 minutes
Call participants:
Kevin Haseyama -- Strategic Planning and Investor Relations Manager
Robert Harrison -- Chairman, President and Chief Executive Officer
Ralph Mesick -- Vice Chairman and Chief Risk Officer, Risk Management Group
Ravi Mallela -- Executive Vice President and Chief Financial Officer, Finance Group
Ebrahim Poonawala -- Bank of America Merrill Lynch -- Analyst
Steven Alexopoulus -- J.P. Morgan -- Analyst
Andrew Liesch -- Piper Sandler -- Analyst
Jacquelynne Chimera -- Keefe, Bruyette & Woods -- Analyst
Brock Vandervliet -- UBS -- Analyst
Jared Shaw -- Wells Fargo Securities -- Analyst
Alex Matters -- Goldman Sachs -- Analyst
Laurie Hunsicker -- Compass Point -- Analyst
